Mind the Gap: Charlotte Mbali Abstract When lecturers seek to improve their teaching skills they may initially demand mere performance tips, but what they can come to value more is understanding processes of learning. This was investigated during workshops for lecturers at three Universities in Uganda at which participants were asked to write down what they hoped to gain from the workshops, and then on completion of the workshops they were asked to write what had interested them most and what actions they planned to take. The responses were coded in NVIVO and analysis reveals that expectations were met and also that many found the workshops gave them new views on learning. Alerted by this attempt to "mind the gap" (between expectations and outcomes), lecturers in other Higher Education disciplines may want to implement similar formative assessment of their teaching (and their students' learning) by coding pre-course statements on expectations as well as in-course or post-course evaluative responses.
Should the participant, in events or courses for improving higher education pedagogy, have expectations fulfilled in the manner of a fast food shopper? Fast food comes in standard quantities and prices. The customer can point to the picture, request the item and rely on it fulfilling expectations. It is not surprising that post-modern consumer societies increasingly favour outcome-based educational programmes, with outcomes carefully defined in advance, and mix-and-match student options, like the cafeteria style of pre-packaged food, all in transparent wrappings, so the hungry customers can see exactly what to choose. Those of us engaged in serving up workshops for adult learning or continuous professional development might sometimes be tempted to confine our efforts to keeping the clients happy by giving them what they expect – and no more. The problem with this is that untutored expectations fall short of the potential vision. What people think they want will not capacitate them for the length of the journey. Higher Education teachers, immersed in their various disciplines, often think that all they need to improve their teaching is a few “tips” and “techniques” to deliver the content better. They are unaware that what is needed first is a re-orientation towards the relational essence of teaching: they have to discover how to relate better to their learners before they can understand the rationale for the various “tips” in better delivery of content. Similar gaps between student expectation and curricular aims are common in many disciplines in higher education. To draw attention to it, I often repeat what a zoology lecturer told one class: his students arrive in Week One delighted to have achieved a place in a course that they anticipate will lead to a glamorous career in the South African Game Parks (safari, the Big Five, adventures in the veld), but he has to gently reveal to them that most of zoology is the study of much smaller wild life, such as beetles. I emphasize that it is the teacher’s job to be aware of student expectations, and to skilfully achieve some alignment along the way between expectations and outcomes. So if that is what I believe, how do I achieve this, faced with an anticipated gap between participant expectations and facilitator (i.e. my own) goals in workshops advertised for the professional development of lecturers? The opportunity came, with some sabbatical weeks in Uganda, to run workshops at three Universities: Uganda Christian University (UCU); Uganda Martyrs University (UMU) and Makerere. None of these had been running pedagogical workshops for lecturers in the recent past. For rigorous research purposes, it might have been considered better to have scheduled the lectures in the same way, and to have utilized similar samples of participants. But this project was dependant on what could be organised in the three institutions in the time available (July 14 th – August 27 th 2005, broadly a vacation period), and the data collection was improvised alongside the primarily pedagogical programmes. These programmes were structured around the well-established themes in faculty development of active learning, peer-work, multiple intelligence, affective impact, and improving the learning environment. (Angelo, 1993; Brookfield, 1995; Chickering, 1969; Finkel, 2000; Gardner, 1993; Weimer, 2002).
Before going into the details of the Uganda project, it is as well to draw attention to certain features of this type of research. It began as an improvised attempt to gather participant responses to workshops. The feedback was required to assess whether it was worth giving further workshops and of what type, at all three institutions. The act of responding was not only to gather data: it was also part of the formative learning of the participants, in “classroom assessment techniques” ( Angelo and Cross, 1993). This write-up of the process is intended not only for those involved in similar faculty development or for those particularly interested in Uganda, but also for faculty themselves, in their various disciplines, who might wish to follow up with closer scrutiny of how their students verbalize their feedback responses. Many institutions have gotten into a groove of evaluating by an institutionally imposed likert scale response sheet, with totals used for promotion portfolios. But many higher education teachers realize that open-end verbal responses give them richer insights into what is going on in the mind of their students. Such responses can be categorized using software such as NVIVO. Categorizing different concepts of student learning has been a long established in phenomenographic research (Marton, Hounsell et al. 1997; Prosser and Trigwell 1999; Richardson 1999; Ashworth and Lucas 2000). Phenomenography differs from phenomenology and other forms of qualitative research in its focus on variation and the claim that the categories noted fill all the “outcome space” of the activity perceived (Trigwell 2000) , as with the numerous studies of “deep” and “surface” learning ( Marton, F. and Säljo; Ramsden, P). Such research has been eagerly taken up and used in pedagogical work with higher education teachers, which shows the usefulness of having just a few descriptive categories around which to structure further analysis. Others, such as Henderson (2003), point out that studies of concepts of learning produces “dichotomous representations.” Investigation into the higher education teachers' constructs of teaching and learning is valuable research that goes alongside faculty development. There are various ways of doing such investigations. The Phenomenographic research into the nature of student learning, as described above, has fed into work of faculty development/teaching and learning units across the world; notably via the 13 “Improving Student Learning” Symposia that have been convened by Oxford Brookes OCSLD since 1993. In so far as the work of these units is staff development, it involves changing participants’ constructs. Thus titles of published reports of such projects specifically reveal this change:
There are diverse research tools for investigating this: Graham Gibbs used the action research paradigm. In describing his own earlier work in staff development, he indicates that although it may have helped lecturers in deciding what to do and not to do, it had “no theoretical underpinning” and “seldom challenged lecturers’ conceptions of teaching.” He has gone on from this to do impact research, via survey methods and an Approaches to Teaching Inventory, to see if the issues promoted by faculty developers actually result in improved teaching as perceived by the students (Gibbs and Coffey 2004). Nicholls (2005) research used Kelly’s Personal Construct theory (1955) to elicit from 20 new lecturers their constructs of key academic roles. She cites Sherman et al (1987) on the progression of teacher development from Stage 1 “when teaching is the giving of information” to stage 4 where “teaching is a complex interaction that is unique and dynamic.” In the case of this research, the workshop facilitator had a prior notion that the lecturers in Uganda would need to shift from an excessively transmission mode of “the giving of information” to a more interactive mode. So this was one of the main objectives of the workshop and a main focus of interest when coding the data. But another feature of research that claims phenomenology as intellectual heritage (Richardson 1999) is that the researcher should be “bracketing out” pre-conceptions or prior assumptions about the activity. As the person coding the data from the comments was also the workshop facilitator (and writer of this article), it is problematic to claim “bracketing out.” But does such “objectivity” matter if the project is primarily action research aimed at improvements? (Guba and Lincoln 1989; Toulmin and Gustavsen 1996) Thus the investigation below can be regarded merely as an exercise in formative evaluation, useful primarily in its particularly Ugandan context. But in publishing such an investigation there is an assumption that readers will be able to discern more general applications, both in faculty development and also in how to do qualitative research in their own disciplines on student expectations and responses
The differences in delivery were as follows: At UCU:
In all, some 56 individuals participated in at least one workshop with a core of about 8 individuals attending many of the topic-related workshops in the third week. At UMU, the same programme was condensed into four and a half days, with a smaller number of participants (27), with a core of about 4 attending multiple workshops. At Makerere, the week available coincided with the start of registration week for the new academic year. One workshop of 90 mins + 120 mins was arranged, attended by some 25 lecturers, from a wide range of disciplines.
At UCU, as participants came into the first workshop they were asked to write down at least 2 things they wanted to learn from the workshop. Tthis was collected at the end of the workshop. Another questionnaire (Appendix 1) was given out at the beginning of the second workshop that asked about the first workshop (see section below). Then, together with the staff development team that was emerging, another questionnaire (see Appendix 3) was devised which was utilized for all the topic-related workshops of the third week. This same questionnaire was then used at UMU and Makerere. The responses were transcribed into NVIVO and then codified. In all, some 26 different expectation nodes were identified (Appendix 4), but most of these had only 1 or 2 instances. Profile counting of instances at these nodes showed:
Obviously this type of profile analysis does not conform to the strict criteria of quantitative research. Also it should be pointed out that the coding was done impressionistically by only one researcher (see Appendix 2 for typical wording extract for nodes relating to teacher-dominant views of learning). But what is revealed above seems to show that the greatest number of respondents (38) arrived at the workshops expecting to learn some new methods or techniques, “tips for teachers.” But further examination of the table reveals that there is an interesting difference between UCU and UMU: whereas at UCU at least 14 couched their expectation in terms of a teacher-dominant view of learning (in the nodes “deliver materials,” “information-giver” and “lecturing”) the UMU participants (12) expected to learn something more “interactive.” It is interesting to speculate the possible reasons for this. Both are private Universities of Christian foundations: UCU has developed from a former Protestant Theological College, and UMU is a Roman Catholic foundation. Do Protestant preaching traditions tend to shape a teacher-dominant view of pedagogy while the RC liturgy-centred worship a more interactive one? Or did the UMU sample contain more individuals (some of them in religious orders) who have more experience of workshop-style sessions of faith development? It should be noted that the method of questioning was entirely open-ended: the respondents put what they wanted in their own words. However, the title of the workshop may, of course, have influenced them. This was especially true of the later topic-related workshops where some respondents simply echoed the title in their expectation section of the questionnaire. It is also possible that some were influenced by the first workshop with its emphasis on active learning to re-phrase their expectations towards more active learning. Even so, it is noteworthy that at UMU they (a smaller group) picked up the inter-active emphasis rather more than at UCU.
The participants were not initially told the outcomes for the workshops. But midway through workshop one, their attention was drawn to a page in the workshop pack where the outcomes of the Introduction to Higher Education Practice (HEP) of the University of KwaZulu-Natal (UKZN) are printed out. The pack itself contained the first 4 session outlines of the version of HEP delivered in 2004 in South Africa. Session 1 has explicit sections on “Class Bonding” with some extended comment on how participants learn from each other, and also on “Modelling” which explains how the facilitators of the sessions would be modelling some of the techniques which are advocated (for example, if what is advocated is getting to know your students, then several ways of getting to know your students were being tried out in the class). To summarize the content of the initial two workshops (Active Learning and Facilitating Learning): Getting to know your students – their names, their expectations, their motivation, their experience of the subject, etc. What is Learning – in three cycles: narrative (each one tells another about a significant learning event); verbal (looking at own definitions of learning) , and “picture smart” (interpreting pictures as symbolic of learning). Learner-centred and active learning “teacher teaching less so that the learner may learn more” transl: Comenius, quoted in Angelo (1993). Some theory
Recommended “Activities” to try out with your own students. The topic-related workshops were delivered in the same inter-active way, with lots of opportunity for participants to share experiences.
Ugandan Universities have not spent a lot of money on books in recent years, as the exchange rate makes books produced in rich countries extremely expensive. This means that the lecturers, at worst, tend to rely on the set of notes that they themselves have formulated from their own time of qualifying. In the mid-1990s, Makerere pioneered a way of increasing its income by opening up double shifts. The government-sponsored students, or those who had achieved the requisite grades to be so admitted, would do the day shift, and then increasingly large numbers of fee-paying students would be admitted for the evening shift. Thus, in order to get a decent salary, most Makerere lecturers do this double shift. And some of the lecturers at the private Universities also teach in other HEIs or in Makerere. This means that their teaching load is such that there is little time for research or further reading. The roads in Kampala are congested and thus moving between institutions can be time-consuming. Although UMU has a core of staff (the religious, mostly, and some senior staff) who live on campus, other staff come out on the bus – some two hours from Kampala. UCU at Mukono is more favourably situated to retain staff with families, but currently some of the younger staff tend to be employed only part-time (which means they have another job as well). Not surprisingly then, reading is a problem for most of the workshop participants. A little test of this at UCU was carried out between the first workshop and the second, which were a week apart. The workshop pack consisted of some 33 pages. During workshop 1, various pages to read had been pointed out, mostly readings not longer than a page, for example the list of HEP outcomes; an extract of Dickens (1854) on Gradgrind; Ramsden on deep and surface learning; a Learning Styles questionnaire AND the 6 pages of summary of Student Development theory from Chickering (1969). The questionnaire given out at the beginning of workshop 2 asked what they had done since the last workshop. Nearly all of them said they had talked about the workshop with colleagues (Africa learns by talking!), but less than half had done any reading at all, not even of the one pagers. Only 2 had read the Student Development theory pages (and one of those was the professor responsible for inviting me). Only about 4 or 5 of the 8 or so who had promised to present an Educational Psychology construct from the list, actually turned up having prepared it, even though it had been emphasised that they could utilise only one source, and via a search on Google if they so chose. Yet several of them, in the questionnaires on the topic-based workshops, expressed a wish to get deeper into the theory. This between-workshop check on learning progress was only possible at UCU where there was week between the workshops. This lack of reading was remarked upon to the UCU core team, and repeated at the workshop at the other Universities, which may have influenced their questionnaire responses coded “theory.” The profile analysis shows that UMU was keen on more theory, and Makerere especially keen on Multiple Intelligences.
Appendix 4 gives the profile coding for all the responses gathered. The first level of each node tree is the responses derived from the Expectations question (Appendix 3 question 3). The sub-nodes are from the outcomes questions (Appendix 3, questions 5-7) Question 5 = “most interesting thing” has a -i suffix to the sub-node name For example: Scrutiny of Appendix 4 reveals that the nodes that gathered the most responses can be put in a table
Some of the gaps under the Makerere column occur because Makerere got much less workshop time: in fact, quality issues were not dealt with at all. Some responses from Makerere under assessment do occur because the workshop there, derived from the HEP materials, featured some explanation of the portfolio as an assessment method and this greatly interested some of the Makerere workshop participants. It is noteworthy that UMU responded enthusiastically to the opportunity to learn from other staff. Makerere, which has crushing numbers to deal with and lecture-dominated delivery modes because of double shifts, were most enthused with the suggestions that students could do more work “learner autonomy.” UCU had a higher score for wanting clarification on materials because the workshop on materials had the highest attendance (n=18) because it drew in participants from the technology sector and had a software demonstration. A higher number at UCU (compared to the other Universities) valued the way the workshops not only advocated certain techniques but also modelled them. UMU scores highly on the many responses where the intention to read up on theory was expressed. However a caveat here – as some individuals attended 6 of the topic-focussed workshops, these 22 responses could have come from just a few individuals systematically stating their intention to follow up on theory each time! It could be posited that what this is showing is something of the Roman Catholic Scholastic tradition to reach for the theory and first principles. 24:2 also shows that these participants were most eager to try out new methods too. Utilizing NVIVO for qualitative analysis of questionnaire responses is not a rigorous statistical research method, but it does shed light on some features in data gathered on the run-for-action research purposes. The total list of responses coded as found in Appendix 4 shows the range of outcomes within the participants – thus 26 nodes of outcomes were perceived in the responses of the 108 lecturers who attended any one or more workshops at any one of the three Universities. It should be noted that this type of NVIVO coding could be regarded as highly subjective. The workshop facilitator, the NVIVO coder and the writer of this paper are all the same person. Did I only code what I was looking for? Have I only picked out for analysis the trends that interested me and which corresponded to the workshop goals? In response to such charges, it should be pointed out that NVIVO does help to objectify textual data because the original verbatim transcribed responses are held in the documents where the coding can be checked by the original coder or by other researchers. After the profile coding revealed which are the most popular nodes, re-checking did reveal some responses that required re-coding with consequent re-adjustment of the profile counts.
Two of the bottom rows of appendix 4 (node 24 and 26) show where there was some correspondence between the expectations of the participants and what was delivered: 38 participants expected tips on methods, and 29 responded on methods either as the thing that interested them most or as something they would try out. The answers under meta-learning, facilitation modelling (n=13) could also be classified here as satisfying expectations for new methods or techniques. Many of those who came to topic specific workshops at UCU or UMU stated their expectations in the terms of the title or topic of the workshop. After the workshop almost the same number (not necessarily the same people) stated the outcome as to what most interested them in terms of that topic, or wanted some further clarification of it. Thus it can be claimed that where there is correspondence between expectations and outcomes, there is client satisfaction. However, the most interesting thing revealed by the profile count is where there are differences between expectations (layer 1) and outcomes (sub-layers). Another scan of Appendix 4 reveals where the participants learned something they had NOT specified in their expectations or now specified a wish to learn:
If a workshop facilitator or professional developer is to be a change agent, then these are the features of this research to focus on. As an experienced faculty developer, I could have anticipated garnering a good number of positive responses for affectivness, for learning from other staff, and for finding out more about students. One of the single most important things lecturers gain once they participate in such teacher development sessions is to realize that they are teaching students (i.e. people) first and their subjects second, and teaching people involves the affective mode, not just the cognitive. If the workshop is designed to be inter-active, then consistently the participants get pleasure from learning from each other. Other items that feature in the list above can be explained from the situation of Ugandan Higher Education: the lecturers are keen to acquaint themselves with methods of assessment, of uses of I.T., and of quality assurance that have not yet reached Uganda. The Personal intellectual development shows a hunger for more time for reading on pedagogy, time which is not readily available to staff who have to do double shifts and/or struggle through Kampala’s blocked or flooded roads to work. It is particularly gratifying to note where participants learned more or different things from their expectations, as gauged in the NVIVO profile count of Appendix 4. These show the growing points in their professional development as higher education teachers. This investigation to compare expectations to stated outcomes confirms the need for faculty developers to provide training which is not directed at Sherman’s stage 1 (methods for improving how to give out information) but rather which is leading participants towards Sherman’s stage 4 (learning as complex interactions). This is why it is important to “mind the gap” between expectations and outcomes. We are trying to shift conceptions of teaching and learning so that most can operate at stage 4.
Angelo, T. (1993) A Teacher’s Dozen AAHE Bulletin Angelo, Thomas A. and Cross, Patricia K. ( 1993) Classroom Assessment Techniques: A handbook for College Teachers San Francisco: Jossey-Bass Ashworth, P. and U. Lucas (2000). "Achieving Empathy and Engagement: a practical approach to the design, conduct and reporting of phenomenographic research." Studies in Higher Education25(3). Brookfield, S. D.(1995) Becoming a Critically Reflective Teacher San Francisco: Jossey-Bass Chickering, A.W. (1969) Education and Identity San Francisco: Jossey-Bass Dickens, C. (1854) Hard Times Houseworld Words Finkel, D. (2000) Teaching with your Mouth Shut Portsmouth, NH: Boynton/Cook Gardner, H. (1993) Frames of Mind: the Theory of Multiple Intelligences New York: Basic Books Gibbs, G. and Coffey, M. (2004) "The Impact of Training of University Teachers on their teaching skills, their approach to teaching and the approach to learning of their students Active Learning in Higher Education" 5:1 pp 87-100 Guba, E. G. and Y. S. Lincoln (1989). Fourth Generation Evaluation. London, Sage. Henderson, Bruce (2003) Teacher Thinking about Students' Thinking MountainRise 1:1 Ho, A. Watkins, D. and Kelly, M. (2001)."The Conceptual Change Approach to Improving Teaching and Learning: an evaluation of a Hong Kong Staff Development programme Higher Education" 42, 143-169. Marton, F., D. Hounsell, & Entwhistle, N. (1997). The Experience of Learning: Implications for Teaching and Studying in Higher Education. Edinburgh, Scottish Academic Press. Marton, F. & Säljo, R. (1984) Approaches to learning. In F. Marton, D. Hounsell & N. Entwistle (eds.) The Experience of Learning, Edinburgh: Scottish Academic Press. Nicholls, G. (2005) "New lecturers' constructions of learning, teaching and research in higher education"Studies in Higher Education 30:5 pp 611-625 Prosser, M. and K. Trigwell (1999). Understanding Learning and Teaching: the experience in higher education. Buckingham, Open University Press. Ramsden, P. (1988) Studying Learning: Improving Teaching in Improving Learning: New Perspectives. London: Kogan Page. Richardson, J. E. (1999). "The Concepts and Methods of Phenomenographic Research." Review of Educational Research 69 (1) Sherman, T., Armisted, L., Fowler, F., Barksdale, M. & Reif, G. (1987) The quest for excellence in university teaching, Journal of HIgher Education, 48(1), pp 66-84. Toulmin, S. and B. Gustavsen, Eds. (1996). Beyond Theory: Changing Organizations through Participation. Amsterdam/Philadelphia, John Benjamins. Trigwell, K. (2000). "Phenomenography: discernment and variation." Improving Student Learning 8. Weimer, Maryellen (2002) Learner-Centred Teaching San Francisco: Jossey-Bass
Appendix 2 Expectations Teaching methods information giver lecturing To be helped to make a lecture more stimulating Delivery Interactive Learner autonomy Outcomes Large classes Assessment Course management Appendix 3 To answer BEFORE the workshop 1. Workshop on 2. What discipline do you teach or department do you work in? 3. What do you hope to learn in this workshop? 4. What fears do you have about this workshop?
5. What was the most interesting thing you learnt during this workshop? 6. What specific actions do you intend to take as a result of this workshop? 7. What points need further clarification for you? 8. What would you like further workshops on? ( tick or add a topic)
* coding discontinued on this node because almost everybody mentioned this |