At the coal face: critical reflections on two tutors’ formative assessment practices under pressure.
Billy Meyer and Penny Niven
Introduction
The context for this research was an Academic Literacy course at the UKZN. The course is part of a broader Access (Foundation) Programme, which is designed to help 1st year, mostly Zulu First language students from impoverished educational backgrounds adjust to the reading and writing requirements of a Humanities or Social Sciences degree in English. We (the writers) are both tutors on this course, sharing the teaching of some 75 ‘Access’ students. During the first semester of 2006 the outcomes of this course were, initially, that the students develop a broad grasp of the notion of ‘genre’ and gain exposure to several varieties of spoken and written genres. Secondly, we focused on writing in one particular genre – the formal academic essay. During the final weeks of the semester, the students were extensively supported in the writing of a first draft, an optional second draft, and then a final version of this essay. The students were required to read and then write from a variety of source materials, describing the role of radio and television in shaping the lives of South Africans before and after 1994. We responded extensively, and, we hoped, formatively, to the first drafts of the essays but only graded the final versions.
CRITICAL ACTION RESEARCH
In this paper, we report on the first stage of a cyclical, small-scale, critical action research project. Action research fitted well with our intentions because it aims to understand and improve on particular aspects of professional, educational practice – in this case, the quality and nature of our written formative feedback on essays. Action research usually takes place within “…a relatively circumscribed domain” (Carr and Kemis, 1986: 205).
The Research paper on Final Exam
Final Exam Study Tips for Biology 400 The Biology 400 Final exam will consist of approximately 75 multiple choice questions. The students will take the test on a scantron. Each final exam period is 90 minutes long and students will get the entire time if needed to take the final. Taking the exam will be a more positive experience if you study the following objectives. Chapter numbers are given for ...
In this case the context is an existing group of 1st year students and their written essays. Besides being an“activist” research model, action research is also “critical” (p 205) because it aims to expose and identify the self-interests and ideological incoherence of educators. By reflecting on why they act in particular ways, the educators can then resolve to act in theoretically stronger ways. In this instance, we as tutors on the course critically interrogated the ideologies informing our formative feedback. Having noted incoherence within our practices, we wanted to revisit the theoretical frameworks for a better practice, and to develop a stronger, more accountable “praxis” (p 190).
Action research is also ‘critical’ is because it aims to be “emancipatory” (p 202), that is to generate real change, bringing “transformations of consciousness” into “transformations of social realities” (p 181).
Furthermore, action research is “participatory” and “inclusive” in character (p 191).
Therefore this project aims, over time, to work collaboratively with a larger group of tutors, involving them in them in the research process. At its best, this type of research develops “self-reflective communities” (p 201).
In a context where there are usually seven Academic Literacy teachers, including some inexperienced Masters and PHD students, across the two UKZN campuses on which this course is offered, a project such as this one has the potential to be mutually developmental, building capacity within the community of practitioners.
Action research involves “spirals” or “cycles” of planning, acting, observing and reflecting (p 186).
This “self-reflective spiral” (p 184) gives evidence of the “dialectical quality of action research: the dialectic of retrospective analysis and prospective action”, (p 185).
ICCT Research Paper
The Proponents wish to recognize the significant contributions of the following people on the preparation and completion of the commitments in providing guidance and support. The Families, Friends and Mentors for their prayers and encouragement. The subject school, ICCT College its registrar personnel for granting access to their records. Lastly to our Lord Almighty, for his guidance and grace. i ...
This paper describes one cycle within the broader framework of an ongoing series of interactions between reflection, theory and practice.
METHODOLOGY
The first stage in the essay writing process was the starting point for this research cycle. Between the first and final drafts of the essay, we swapped our students’ scripts, initially with a view to standardising our marking criteria for the summative assessment at the end of the semester. This gave us unexpected opportunities for observing, in detail, each other’s styles of formative feedback, because the students were required to submit their first drafts alongside their final versions. In fact, one of our criteria for the summative assessment at the end of the process, was making a decision about the depth and quality of the changes which the students had made between their first essay and their final draft in response to our formative feedback. Engaging with this process we started to note the surprisingly different ways in which we gave formative feedback. We began to consider and discuss these differences and wonder whether they mattered – whether our comments were really aligned with the courses’ outcomes. We also wondered if our responses truly conformed to any of the substantial theories about, or research into, good formative feedback – we hoped they were, but began to doubt it. Were our ‘espoused theories’ truly our theories in action — Argyris – extend here).
We speculated about whether our comments were experienced as helpful by the students? Which interventions were effecting real changes in the students’ final versions? What were the underlying principles informing our responses, if indeed they were ‘principled’ at all? Within these professional self-reflections, we saw that, in a spirit of collegiality, there was a rich potential for peer- and self- evaluation of our practices, ways in which we could learn from each other and from our students. We saw that there was, indeed, potential for a small action research project within a supportive, self-reflective community of practice (Rowan, ref?).
On the basis of our observations and reflections described above we set about planning the first stage in our research cycle. Carr and Kemis describe the “action moment” of the research cycle as “… a probe into the future” (1996: 185) and plans, they say, are “…retrospectively constructed on the basis of reflection,” (ibid).
Modern Student Achiever Good Time
Modern Student When it comes to being a student there are typically three ways that you can go about taking on work: the over achiever, the median, and the procrastinator. Everyone wants to be the over achiever, but not all have the initiative to do so. The type of student you are can show a lot about how far you will succeed in life. The over achiever is the student that does more than what he or ...
So initially, we revisited some of the research into formative feedback and theories about which feedback practices are effective – these are described in the Literature Review below. We then attempted to identify the true, underlying motives driving our formative assessment. In order to do this we did some free writing, a technique recommended by Elbow (1998) for …. (Explain more about Elbow here).
(See Appendix B).
We then returned to the students’ first essay drafts and recorded many of the comments we had made on them. As we proceeded, we began to understand our comments in terms of both our espoused ideology and the underlying ideas which were also powerfully influencing our practices and decisions. We noted that two clear contrasts emerged between; firstly, our own different feedback styles, and, secondly, between what we metaphorically termed our ‘above-ground’ practices (based on the good principles enumerated above) and ‘underground’ practices based on the free writing in Appendix B.
LITERATURE REVIEW
The following are some principles of good formative feedback – our espoused, ‘above-ground’ ideology. They are obviously not discrete, separate entities – in practice they are much more nebulous – but they are numbered to make our data easier to analyse and refer to. They are based on reading extensively about the kinds of feedback practices that support learning – Black and William, (1998: 48); Ivanic, Clark and Rimmershaw, (2000: 51 – 65); Weeden, Winter and Broadfoot, (2002); Torrance and Pryor, (1998); Boud, (1990); Luckett and Sutherland (??); Elbow (1997); and Ofsted, (1996).
1. Good formative feedback builds and protects students’ self-esteem and confidence. It should be task-involving (focusing on the writing) rather than ego-involving (focusing on the writer).
2. The criteria for evaluating the task must be shared openly with the students at the outset of the assignment. Tutors must make sure that students fully understand the criteria.
3. Feedback on the assignments must target only these pre-planned criteria: extraneous issues or superficial errors can be temporarily ignored.
The Term Paper on Guideline For Good Clinical Practice
INDEX 1.Introduction to Clinical Trials3 2.Clinical Trial Protocol5 3.International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use6 1.1.Introduction to GCP6 1.2.Sections of GCP Guidelines6 3.2.1.1Section 1- Glossary of various terms6 3.2.1.2Section 2- Principles of ICH-GCP7 3.2.1.3Section 3- Institutional Review Board/Independent Ethics ...
4. Feedback should occur quickly if it is to be helpful to the student.
5. Feedback should ‘feed-forward’ – it should inform future drafts or tasks. It is assessment for rather than of learning. It should give constructive, realistic, staged advice on how to improve the next draft or assignment and how to close the gap between what they wrote compared to an ideal answer.
6. It should identify misconceptions. Therefore, it should be meaning or content focused, identifying the substantive issues, and being selective about which surface errors to correct.
7. Feedback should develop a sense of partnership with the student: it will be dialogic and democratic. Facilitative, supportive, collegial feedback deconstructs the image of the tutor as powerful, all-knowing judge. ( Ivanic et al, 2000)
8. Elbow (1997) says that feedback in form of “descriptive, observational responses…” helps students to gain a meta-cognitive awareness of their own writing and thinking processes.
9. Good feedback avoids “final vocabulary” (Boud, 1990).
Comments such as ‘disappointing’, ‘you can do better than this’ or even ‘excellent work’
10. It also avoids overgenerous and unfocussed praise because it can actually reinforce under-achievement. Black and William’s research into feedback (1998) says that feedback that provides only a grade, or ‘praise and a grade’ not only fails to improve students’ writing, but also sometimes actually deteriorates it.
FINDINGS:
When we came to turn the focus of our attention on the nature of our own written feedback, rather than on our students’ writing, we ascertained that at least some of our practices did conform to the “Good Principles” we outlined earlier. Others plainly demonstrated our ‘raw’, underground principles in Appendix 2. We will deal firstly with examples of our ‘above-ground’ praxis and then of our ‘underground’ practice.
Strong Practices That Support Learning;
In keeping with our commitment to good process we could confirm our adherence to basic principles of good formative feedback to the students on their essays. The assessment criteria for the essay were shared with the students (Good Principle 2) and each essay draft was accompanied by an assessment sheet (Appendix 3) In addition our students had the first drafts of their essays returned to them within a week (Good Principle 4).
The Homework on Good Student And Good Teacher
Good teachers have a good qualifications and subject knowledge; they are organised, energetic and have empathy and good communication skills. This is obviously important for good education. However, it is really the diligence and intelligence of the student that determines whether learning will be successful or not. Although every student benefits from good teachers, only a fraction of a ...
We also found very few comments that were unrelated to our criteria. (Good Principle 3) However, we did note that our criteria were so broad and all-inclusive; it would have been difficult to find comments that were extraneous.
But in addition these basic good practices we have observed a number of particular strengths in each of our personal styles. Penny’s Strength is her desire to build her students’ self esteem, and to be ‘descriptive and observational’ (See Good Principles 1 and 8) in her assessment of their performance in learning the academic essay genre. A few examples will suffice to show her style: “Nice summing up sentence here” “Better here: more relevant”“This is a nice introduction” “You have tried really hard with your linking language and nearly succeeded” “Nice transitional sentence that leads the reader into the next section”
Helpful feedback should be meaning and content focused. Therefore it should identify substantive, conceptual issues in the students writing and help them to focus on the content of the essay readings and clarify what they mean (Good principle 6).One of the strengths in Billy’s feedback is his attempt to get students to reflect accurately the content of what they had been reading and stretch their conceptual understanding of what they are writing about:
“What kind of places did people live in and why did the government want to keep them there? You must explain the government policy here in as much detail as you can”. “Theatre for Development is not radio or TV. The point is that the producers of these SABC programmes were inspired by the methods of Theatre for Development”.
“You must have two separate paragraphs here. One should explain why Soul City is effective and the other should focus on the violence and gangsterism in Yizo Yizo and say why this programme is less successful”..
We both show some strength in giving genuinely formative feedback. There were numerous examples of the ‘feed-forward’ principle (Good Principle 5) aimed at helping the students to improve each draft of their essay. The focus of Penny suggestions was often aimed at helping the students build a more coherent structure for their next draft:
The Essay on The Pleasure Principle
The Oxford Dictionary defines pleasure. It is a feeling of satisfaction or joy. It is enjoyment. It is a source of pleasure or gratification. It is formally a persons will or desire. Finally they define it as sensual gratification. Now if we put these all together in one sentence, pleasure is something that brings an overall feeling of satisfaction and gratification while fulfilling desires.If ...
“You need to reorder your paragraphs. Discuss all the pre-1994 paras together and keep all the post-1994 paras together”.
“Shouldn’t this paragraph be placed earlier in your essay?”
“You could split the paragraph here because you are moving on to a different topic.”
However, Billy’s ‘feed-forward’ comments also reflect strong conceptual focus we noted earlier: “Make sure you are using your own words here. See also my suggestions for where you need to explain more’.
“You need to rewrite this so that it answers the question: it must explain how the SABC were trying to influence young people using popular soap operas”.
“Your first paragraph needs to explain in more detail how the SABC supported the Bantustan policy”.
Feedback Practices That May Not Support Learning
However, our ‘underground’ motivations were also in evidence: sometimes our comments may have failed to “build and protect students’ self esteem and confidence” (Good Principle 1).
Billy was very forthright on a number of occasions when the students’ conceptual grasp of the readings and organization of information tested his patience.
“This is wrong! The point is that by questioning tradition the plays were questioning the whole idea of separate traditional cultures and the idea that people should live in traditional homelands”.
“You have quite some work to do. I expect more sophistication in your argument”
“Your information is totally disorganised and the same points are repeated again and again”.
“You just write down any idea that comes into your head without thinking about how it relates to the topic or to the rest of the ideas in the paragraph”.
Significantly, in terms of Penny’s underground motivations, we found only one strongly negative comment we found in her feedback. “I don’t like this sentence….” This was in response to a sentence in a student’s conclusion which ran: “After 1994, Yizo Yizo and Soul City succeeded to educate, develop, motivate and bring entertainment to all South Africans: I am saying that because the community who watched and listeners of radio stations are the witnesses”. Penny’s comment was an irritated reaction the generic inappropriateness of the second part of the sentence. More often, Penny’s ‘underground motivations’ led her to comment on students essays using “Over-generous and unfocussed praise” (Ofsted, 1996: Good Principle 10) and “Final vocabulary” (Boud, 1999: Good Prinicple 9) which do not really help the students see how their drafts could be improved and may give them a false sense of their ability as essay writers.
“A good effort, but this could be improved in the final draft” “Some nice paragraphs but I think your essay could be more coherent. You also need a clearer thesis” “You tried hard and this is a really good effort!” “Apart from language errors, this essay is quite strong. But there’s room for improvement! Good luck!”
In a context in which we were giving formative feedback there were a number of examples which could be interpreted as summative comments, such as:
“The second half of your essay is much stronger than the first half.”
“This is a good essay although I have suggested a number of improvements”
A number of Billy’s essay had “This is a fail” written on them, or “Pass, but you need to do a lot more work”. This is in keeping with Billy’s free writing: “I am not afraid to let students fail at the drafting stage …because they can be lulled into a false sense of security.”
We think it is also significant that, in keeping with our ‘underground’ ideologies expressed above, we found almost no evidence of Good Principle 7 in our feedback. This principle sees the responder in a partnership with the student, dialoguing with her in a supportive, collegial way like the following example.
“Have you said enough about whether their influences were successful or not? Do you need to point this out more explicitly?”
Rather, we attempt to help the student to “mushfake” (Gee 1996) their entry into generic forms. In the following example, Penny offers the student language to help him express his meaning in a more convincing, academic way although the student’s original meaning is reasonably clear. In the final sentence of his conclusion one student had written: “In my point of view media is good especially for education but the main problem is misconception and propaganda.” Penny had suggested he change this sentence to: “(Therefore in my opinion)( the) media (can be)( a positive influence,) especially for education, but the (…) problem is (that it can cause) misconception(s) and propaganda (as well)”. (All the underlined parts in brackets are what Penny suggested he add or take out of his original sentence.) Penny also gave several ‘extraneous’ comments that were not central to our assessment criteria, (Good Principle 2) but they are in keeping with her drive to help students writing ‘look’ better:
“It is always good to write out the essay topic in full, word for word”
“Find another word for ‘discuss’ in this paragraph” (This student had used ‘discuss’ three times in a paragraph: this didn’t affect meaning but is a stylistic issue)
“This vague word should be avoided in academic writing”. She was referring to the word the students repeated use of the word ‘thing’ instead of being specific. Billy’s conceptual focus noted earlier led him to provide strong directives instead of dialoguing with the students:
“Using your own words try to explain what Radio Bantu was trying to do in order to support the apartheid government. Begin like this: “The first example of how the SABC tried to influence the lives of a group of SAs was IsiZulu and IsiXhosa dramas on Radio Bantu in the 1970s…”
We do however need to recognise that we are dealing here with a course aimed specifically at assisting many of our students to deal with the linguistic and cognitive under preparedness, which are the legacy of apartheid education and the continuing crisis of education in both South Africa and the wider world. This context may perhaps necessitate more overtly directive feedback, which is less about partnership and more about building capacity and identifying misconceptions (Good Principle 6).
However where a student does show signs of being ready for further exploration an dialogue we do not shy away from calling them to dialogue with us and to draw connections between readings that may not have been fully explained in the classroom context.
If you read Mistry at this point you will see that her comments about TV shows for Indians like ”Eastern Mosaic” and even Bollywood Movies, show a similarly distorted view of wealth in India. Mistry goes on to write about Youth Identity and how to choose clothes based on the media, which also fits with Bynoe.
CONCLUSIONS
It seems that our praxis is entailed within a number of conflicting theories about what constitutes good, formative feedback. There seems to be a ‘syncretism’ at work: sometimes we work in one mode and sometimes in another. There seem to be ‘underground’ ideologies while we are at the coal-face, and ‘above-ground’ principles which are theoretically sounder and more ‘tutored’. In the heat of responding to some 40 essays within a working week, our instinctual, untutored responses became more evident, and there were even some examples of unfinished sentences, confusing abbreviations and unreadable scrawls in our feedback, clearly generated by the pressured circumstances.
We discovered that while we do share many of the ‘above-ground principles’ represented by the ‘Good Principles’, and much of our better feedback is directed by these ideas, we have distinct personal agendas as well which drive some of the weaker aspects of our feedback styles. In Penny’s feedback there is a tendency to over-generous, unfocussed praise, and a failure to address substantive, cognitive issues. Whilst Billy’s feedback is more cognitively challenging and is meaning-focussed, it can be less clear and concise, and sometimes inclined to expresses personal irritation. We are both, in different ways, strongly directive. Billy has a tendency to be prescriptive – a tendency partly driven by a strong model of Genre theory. The tone of his responses is neither collegial, nor democratic, and his feedback assumes a significant power differential between himself and his students. Penny’s directives may seem less arrogant, sometimes couched as questions, but are often driven by a sense that the students need some superficial skills and competencies that will help them to fake academic literacy, and therefore help them to survive the complex, irrational demands of the academy. It is possible that what she assumes is an ‘empowerment’ agenda, is an assumption that the students cannot be trusted to make meaning on their own, in their own way. Billy assumes that as the students’ find what they mean; the language will follow naturally and automatically. This could signify that he is more influenced by ‘expressivist’ models of learning than he might assume. Whereas Penny is more influenced by the models of writing that are technical and instrumental – a ‘skills’ approach. These are uncomfortable insights.
At this stage in the cycle of our research, we are only dealing with our ‘input’ into our students’ written texts. In the next stage we will address the students’ ‘uptake’. We will have to interview them about the effects of the input and we will have to look more closely at their second drafts to ascertain what was helpful and what failed to help their writing. We will also look more deeply into our motivations perhaps taking a phenomenological approach to uncovering the development of our praxis through our experience as teachers
Questions arise, for which we will need to do more research.
o What really counts in our feedback?
o How much of our feedback is wasted?
o Does it really matter that our styles are different?
o What is the role of the power differential between the students and us?
o If we gave less feedback, and focussed on fewer issues, would it be more helpful?
o How can we encourage each other, and our colleagues, to provide richer, more careful feedback?
o What practices should we retain?
This has been a consciousness raising exercise. In some ways we have been encouraged, but in other ways we have been appalled by the conflicts we have noted between our theory and our praxis. It has been a collegial experience – one which we hope to share with our other colleagues in due course, building a stronger more coherent “community of practice” within our department.
APPENDIX B: THE TUTORS’ FREE WRITING
Tutor 1 wrote: “What I am really interested in getting the students to write clearly. For me, incoherence is the enemy. I hate muddle and confusion. So I think hard about what I think they are trying to say and offer them the language to try and say it. So I probably do offer more linguistic, structural help than I should. I want to help them to ‘dignify’ what they are saying so that it sounds more convincing. I am powerfully motivated by empathy. I have a strong sense of the student’s vulnerability and I really want the students to feel better about their writing and feel that have a place in the university. I often find myself considering ‘ego’ issues while I am giving feedback. So I am perhaps more ‘ego-involving’ than I should be and perhaps that is not helpful. I have a sort of ‘encouragement is all’ mindset and am very anxious about causing hurt or offence. This probably means that I tend to praise too much, making bland, global comments. If I am honest with myself, I think that don’t really see myself as the students’ ‘colleague’. There is still a strong power differential in my mind. I want the students to learn a way of writing in the university that empowers them to ‘make it’ and helps them to ‘mushfake’ convincingly. I see myself as having a strong obligation to help them get by. So there’s little discussion about possibilities or alternatives. I tend to say: this is the way to do it, do it, and you will survive. I think this is what motivates my comments”.
Tutor 2 wrote: ‘Although I share a commitment with (Tutor 1) to essay process, I have perhaps been more strongly influenced by genre studies and the necessity to strongly direct students into the correct generic forms expected in academic discourse. To this end, I find that the shortness of time in a semester long program leaves little space for the niceties of self-esteem building. I want to be explicit about what is wrong with the essay and to stimulate the students cognitive process by asking questions about what they have written, redirecting them to the reading to find more information and curbing their natural instinct for plagiarism. At times, I know that combining this with corrections of grammar and structure can be overwhelming for the students so at times I am willing to sacrifice grammatical correctness and a measure of coherence in my feedback in the hope that as the students’ thinking becomes clearer these problems will sort themselves out. I can be quite hard on the more able students because I want them to push themselves. I am also not afraid to let students “fail” at the drafting stage because I believe they are strongly motivated by marks and can sometimes be lulled into a false sense of security if they think their first effort was good enough to pass’.
