Jim Christophers and Chris Bryan of the college of St Mark and St John examine the problems of assessing impact and present a model that takes account of the individual and the wider community.
Background
The South-West Initiative for Training (SWIfT) programme is a TDA funded partnership between two higher education institutions (HEIs), the College of St Mark and St John (Marjon) and the University of Plymouth, and 10 local authorities. The SWIfT (Marjon) programme is one of flexible distributed learning, meeting the needs of teachers whilst operating in an applied context. All teachers who participate in the programme carry out a needs analysis and, through discussion with their university approved tutor (UAT), devise a plan of postgraduate study and research to address their needs. Following successful completion of their assignment the participant has to indicate formally how their studies have both impacted on their practice and improved pupil performance. The successful completion of a PG Cert (60 credits) and a PG Dip (an additional 60 credits) allows participants to progress on to a dissertation leading to the award of an MEd.
Impact and the PPD bid A fundamental requirement for obtaining postgraduate professional development (PPD) funding is that evidence is provided to show that the programme makes a positive difference to professional practice and pupil performance.
Fortunately, evidence to suggest that the SWIfT programme does make a difference is not difficult to find. Comments such as ‘I have become more reflective’, ‘My research has shown the need for formative assessment at A-level’, and ‘The use of ICT is now embedded within my teaching of numeracy’, have been written recently by teachers on their impact report forms. Similarly, comments on UAT module report forms indicate that the work carried out by participants has led, for example, to ‘improved writing ability and increased self-esteem of pupils leading to enhanced SATs results at KS2’.
However, it was felt that merely to assert that there had been impact was not rigorous enough and that, therefore, the nature of impact evidence should be explored further. Hence, in September 2005 a conference was organised for UATs across the south-west to explore, among other things, the meaning, identification and collection of impact evidence. Central to the conference was the role of the UAT in this process.
During the conference the UATs worked in small groups of three to four to explore the type of evidence which could be used to exemplify impact evidence in the following areas, arising mostly from the TDA bidding specification: l improved knowledge and understanding l improved pupil performance l improved practice l research and problem solving l challenging barriers to teacher participation
l unintended outcomes.
The outcomes from this activity form the stimulus for this report. In very general terms it was observed that impact makes an impression, that makes a difference, and is therefore characterised by change. Also, one should not assume a fixed, deterministic notion of impact but appreciate that context will be an important factor. Impact evidence will be wide-ranging, containing breadth as well as depth.
Consequently, when seeking evidence of impact one should not be seduced by ease of access to convenient data and oversimplify what is a complex task.
However, it was acknowledged that any notion of impact should encompass improved classroom practice, pupil achievement, school improvement and effects on the wider community and that impact evidence should recognise both medium and long-term changes.
PPD in SWIfT
SWIfT’s approach to PPD is founded on the principle that participants will develop most effectively through self-reflection on personal practice, particularly where that is supported by a critical friend. Philosophically it lies within the social-constructivist theoretical paradigm so eloquently expressed by Vygotsky, who emphasised the role of the ‘more knowledgeable other’ within a dialogue taking the learner into a ‘zone of proximal development’, offering a level of challenge set appropriately beyond their comfort zone and thereby ‘adding value’ to the development experience.
In practice, the process begins with the identification of an aspect of an individual’s and/or group’s and/or school’s practice which they find intriguing or fascinating. With the help of a critical friend this is developed into a project, normally structured as an enquiry founded on empirical work, which seeks to shed light on policy, practice and experience within the selected aspect.
This reflects the views of Lawrence Stenhouse who, referring to the earlier work of Hoyle, advocated that the work of teachers should be informed by an ‘extended professionalism’ characterised by, ‘a capacity for autonomous, professional self-development through systematic self-study, through the study of the work of other teachers and through the testing of ideas by classroom research procedures’ (Stenhouse (1975) p144).
At the heart of teacher professionalism, when it is presented in these terms, and also characteristic of postgraduate study, is criticality, a disposition toward questioning, holding teachers in a perpetual state of enquiry – most of it implicitly functioning subliminally.
The Marjon framework of PPD offers opportunities to make explicit what has been implicit and to articulate what has been subliminal by asking direct questions of aspects of one’s practice and engaging in a systematic pursuit of answers.
The role of the university approved tutor The role of the UAT is central to the SWIfT (Marjon) programme. All UATs have a Masters degree or equivalent, together with a background in education. Their role is to act as the critical friend to participants, encouraging reflective practice and guiding them through the requirements of Masters level work; they also comment upon and assess assignments. Brighouse and Woods (1999) define a critical friend as one who ‘asks questions which are increasingly focused, but speculative rather than judgemental’ and ‘sees strengths as well as points for development’.
Pollard (2005) develops this concept further by suggesting that a relationship designed to develop the reflective practitioner should be facilitated by a ‘co-enquirer’, who encourages the participant to take responsibility for their own learning whilst at the same time recognising the complexity, dilemmas, consequences and contexts underlying professional decisions. Thus the relationship is not ‘top down’ with information passed down from an ‘expert’ to the ‘recipient’, as characterised by many traditional Inset courses, rather it is one where the UAT adopts a questioning stance, challenging and opening up the mind of the participant to reflective criticality.
A significant dimension of the dialogue between the teacher and their UAT relates to the description, explanation and examination of impact. The model presented on the opposite page attempts to demonstrate how this may occur.
Measuring impact? Measurement occurs at a moment in time, but impact that is embedded and sustained possesses a temporal dimension, a life beyond the moment.
To paraphrase an observation much used by educationalists when referring to teaching, ‘PPD affects eternity; no one knows where its influence ends’.
Contemporary schools are ‘data rich’, offering a number of opportunities to gather evidence data which can be quantified in very precise terms. However, simply quantifying impact would be reductionist, losing the aesthetic richness of the concept. Indeed, the TDA recognise that different providers will offer very different information as evidence of impact.
Consequently, there will be a need to be flexible, but not floppy, in seeking to identify and articulate the nature of the impact and to be able to do this with reference to evidence which has been obtained systematically and which is both valid and reliable.
Attributing causality, of course, presents its own challenges. However, it is taken as read that whatever aspect of impact is illuminated, it will contribute to the enhancement of the educational experience and performance of pupils.
The model At the centre of the model is the PPD Project. It is here that the critical friend, the Marjon ‘University Approved Tutor’, engages with individuals and/or groups. Radiating from the project are the six impact criteria arising from the bidding specification presented by the TDA. The potential for impact is then set out in respect of the individual and the wider professional community. Evidence gleaned from the UAT’s participation in the project and its assessment will provide valid and reliable feedback about the impact of the experience on the individual.
An important premise of the model is that this will be a catalyst for impact in the wider professional community, so there is a flow from individual impact to wider impact. The project, therefore, represents a pebble dropped into a pond and the resulting ripples flow out through the individual to the wider community. Evidence of this wider impact will come from a range of sources, including the project, and this is represented in the outer circle. So there is a notion of wash and backwash, the ripples reaching the bank and being reflected back toward the centre. In this way a dynamic is created as the development process obtains a life beyond the individual; something which begins with a person addressing a focus at a moment in time possesses a potential for impact which is as far-ranging as it is deeply personal.
Issues arising from the model Within the model the identification, explanation and examination of impact becomes an integral part of the professional development process, recording progress and informing future work. The particular focus of PPD begins at the centre and works outwards. However, the model presents an experience which, in reality, should be viewed holistically as every part of it is interrelated with every other part. What is presented so neatly, clearly and deterministically is actually more messy and amorphous. The TDA criteria facilitate a classification of the dimensions of impact which provides a framework within which to report on principles, processes and experiences. However, this represents but one conception of PPD. Further, those esoteric and extemporised encounters, like casual conversations in the staffroom, for instance, which inform and enrich the professional development experience, should not be devalued.
Professional development is an intrapersonal experience, it has to be owned and internalised to be effective. Even where the individual participates collaboratively in a group project, professional change can only take place within the individual. Indeed work submitted for assessment has to be attributable to the individual. The role of the UAT is fundamental at this point because, by challenging, questioning and taking the participant beyond their comfort zone they enhance the potential impact of the work on the individual, the pupils with whom they work and the wider professional communities and networks within which they participate.
This is the particular contribution that accredited PPD makes to the enhancement of the quality of education more generally. Articulate, critical, reflective practitioners are more aware of their own practice and better able to describe and explain it to others. The critical examination of practice impacts on its quality, enhances the experience and achievement of pupils and raises standards.
The workforce participating in formal education is being remodelled with extended and full-service schools engaging with their communities in many more ways than they did previously; the ‘Every Child Matters: Change for Children’ initiative brings together a range of people involved in the education and care of children. All this is reflected in the recent metamorphosis of the Teacher Training Agency into the Training and Development Agency and needs to be accommodated in how the term ‘teacher’ is to be interpreted.
While the model presented tends to focus on those traditionally leading learning within formal educational settings, it can and will adopt a wider remit as it is translated into the contemporary world of professionals and paraprofessionals charged with the education and care of children more generally, and this, of course, will present a new set of challenges.
‘Any notion of impact should encompass improved classroom practice, pupil achievement, school improvement and effects on the wider community’
‘The project, therefore, represents a pebble dropped into a pond and the resulting ripples flow out through the individual to the wider community‘
Making a difference – a model illuminating the nature of the impact of postgraduate professional development
References
l Brighouse, T, and Woods, D (1999), How to Improve Your School. London: Routledge. l Pollard, A (2005), Reflective Teaching. London: Continuum.
l Stenhouse, L (1975), An Introduction to Curriculum Research and Development. London: Heinemann.
Editor’s comment: not only about numbers
There really is no escaping this word ‘impact’.
Jim Christophers and Chris Bryan point out that schools are ‘data rich’. This, they say, does not mean that all evidence for impact must be presented in quantitative form. The concept of impact should reflect the complex, rich, uncertain, unexpected but highly valuable professional learning of teachers. Throughout the country there are people like Jim and Chris gathering, sorting and making sense of professional learning and the impact it is having upon the practice of teachers and the learning experience of children. I believe that not one of them would say that impact is only about numbers.
CPD Update intends to bring to the attention of readers further examples of school-based research into CPD.