Critical Reading Outcomes and Literary Study in a Problem-Based Learning (PBL) Literature Course
Jeff Sommers
Department of English Miami University Middletown
Middletown, Ohio, USA
Helane Adams Androne
Miami University Middletown
Middletown, Ohio, USA
Ellenmarie Wahlrab
Miami University Middletown
Middletown, Ohio, USA
Angela Polacheck Miami University Middletown
Middletown, Ohio, USA
Abstract
What happens when students move through apprenticeship to mastery as literary readers? How can we observe that growth? This research project describes the degree and kind of critical literary reading responses demonstrated by students in a problem-based learning (PBL) introductory literature course. By using a rubric we designed (A Map of Literary Reading Responses), we learned that students demonstrated that they were reading critically in several desirable reading behavior domains. Some evidence suggests that more mature students (i.e. upperclassmen) may have been more successful as critical readers, but this descriptive study illustrates that first-year students and students with no previous literature course experiences also succeeded in demonstrating desirable critical reading behaviors. This study suggests that the PBL approach has a place in literary study and sketches out some further avenues for research into critical reading response.
Critical Reading: A Descriptive Study
English professors have long acted upon the belief that there are significant differences between inexperienced and experienced literary readers. The heart of these discussions is that inexperienced readers are passive consumers of the texts they read while critical readers play an active role in constructing meanings of those texts (see Blau, 2003). When Dorothy Sarbin notes that “Learning in college is a kind of apprenticeship, in which the student learns by doing, often by imitating those who are already proficient” (2003, 1), she describes the landscape of most introductory literature courses, courses in which relatively passive, inexperienced consumers of texts develop into more active, critical, and experienced readers of texts. But what happens when students move through apprenticeship to mastery as literary readers? How can we observe that growth? We decided to design a research project that would make that process visible, and as we did so, those questions eventually focused on the critical reading engendered in a very specific kind of literature course, one based on a problem-based learning (PBL) pedagogy.
What Blau and Sarbin are discussing is not so much the reading process itself as the outcome of that process, the observable responses of readers through their work in a literature class, whether through their contributions to class discussions or their written commentary in journals, essays, or examinations. The distinction between the process of reading and the outcomes of reading is central to our project. We made the decision that rather than investigate how students comprehend literature, we would focus on how they demonstrate what they have comprehended. Over four decades ago, James R. Squire made the same distinction when he argued thatteachers needed to develop techniques for assessing the quality of students’ responses to literature apart from measuring literal comprehension. “By analyzing individual oral and written comments regarding literary selections, the teacher may obtain some rough indication of the nature of such reactions,” he concluded (1964, 56). Researchers have been examining the reading responses of student readers ever since Squire’s report was issued (see Kintgen, 1983; Rogers, 1991; Peskin, 1998, among others).
Literary study offers a disciplinary framework for literature instruction, a point Lucy Cromwell makes when she urges that literary instruction should provide students with “tools” to read analytically with the objective of cultivating a “‘literature mind-set’” (80). That “mind-set,” as she defines it, requires students to move beyond literal decoding of a story’s plot to a consideration of the writer’s choices. “Looking at the world from a background in literature study is an inductive way of responding” (80) that requires students to pay close attention to detail and to acquire the habit of providing evidence to substantiate their inferences. “As instructors,” she concludes, “we systematize the reader’s role by evaluating the validity of students’ responses” (79). While Cromwell’s evaluation is most likely assigning grades and providing formative response to student work, we can extrapolate from her argument that one way to assess the degree of “literature mind-set” in a class would be to evaluate the students’ responses. This study thus set out to complete that evaluation in hopes of providing a rich description of the forms of critical reading outcomes the students have manifested.
Our intent in studying these critical reading outcomes is thus a descriptive one, as presented by Pat Hutchings of the Carnegie Foundation for the Advancement of Teaching. Hutchings has proposed a taxonomy of questions that researchers in the scholarship of teaching and learning might pose:
- What works? This is a measurement or argumentation question, the intent of which is to make a case for the effectiveness of a particular pedagogical approach
- What’s possible? This is a question that focuses on design, examining the consequences of making changes in pedagogy
- What is? This is a descriptive or analytical question intended to learn what is actually occurring in a given learning situation. Hutchings elaborates by
explaining that “Here the effort is aimed not so much at proving (or disproving) the effectiveness of a particular approach or intervention but at describing what it
looks like, what its constituent features might be” (3).
We thus designed a study that would allow us to answer the question, “How can the critical reading responses of students in a literature course be made visible and assessed?” situating our work in the forty-year tradition of reading studies dating back to James R. Squire. What follows next is a description of the course we studied and the design of the study that we developed.
A Literary Problem?
Professor Bart D. Ehrman, Chair, Department of Religious Studies, University of North Carolina, was so upset that he recently published Truth and Fiction in The DaVinci Code: A Historian Reveals What We Really Know about Jesus, Mary Magdalene and Constantine (2004). The DaVinci Code by Dan Brown is a best-selling novel, a thriller about Leonardo DaVinci and Jesus Christ. But Ehrman is not the only reader who is upset about the book: Christian groups have denounced it; the country of Lebanon has banned it. The issue at stake is the novel’s historical accuracy.
In a recent interview, Professor Ehrman was asked, “Why do the inaccuracies in The DaVinci Code matter? It’s a novel, after all.” He responded, “The first page of the book has a fact sheet on which Dan Brown says that everything is factual except the fictional characters. He makes an explicit statement in which he says all descriptions of artwork, architecture, documents and secret rituals are accurate. The point is that he claims to be building on a factual basis, and people get misled” (“The Dan Brown Code”, 2004, 2).
This news article caught the lead author’s (Sommers) attention (T his research project was developed under the auspices of the Carnegie Foundation for the Advancement of Teaching during the lead author’s year as a Carnegie Academy for the Scholarship of Teaching and Learning scholar. The research team was formed as part of that project.) because he had taught a course on historical fiction in the Spring 2004 that addressed the very issue on Ehrman’s mind: how are readers supposed to read historical fiction? The course focused on an academic version of that question: “How do students learn to read historical fiction in a literature course?” Let us briefly trace the development of that latter question and why this particular course became the one we used in our research study.
After more than twenty years of teaching literature courses at an open-enrollment, two-year regional campus of Miami University, the lead author’s teaching philosophy had come to emphasize the development of learning processes more than the acquisition of information in those courses. In other words, the goal of literary study is to promote critical thinking through critical reading to a student population which consists predominantly of inexperienced literary readers, rather than to impart information about literary history or critical approaches to reading. Of course, while a literature class that emphasizes close reading, class discussion, and frequent writing probably assists students in becoming active critical readers, problem-based learning (PBL) offered a teaching approach that emphasized the need for students to read more critically. Chet Meyers argues that "Teaching a framework for analysis will be in vain unless students have the motivation to engage in critical thinking. To develop this motivation, students must actively struggle with real problems and issues–and see their instructors doing the same" (1986, 8). When a central problem is posed in a literature course, the reading takes place within a carefully designed context: solving the problem posed by the course. PBL establishes apurpose for the critical reading and thinking in a literature course.
PBL is “focused, experiential learning organized around the investigation, explanation, and resolution of meaningful problems. In PBL, students work in small collaborative groups and learn what they need to know in order to solve a problem” (Hmelo-Silver, 2004, 236) although it has not been employed extensively in literary study ( Bill Hutchings and Karen O’Rourke have been experimenting with the use of PBL in literature courses, arguing that students should be required to engage in the actual basic activities of the discipline of literary study, emphasizing the “active, creative engagement of the reader to meet the creative power of the literature,” 2002, 75). In the fall 2002 the lead author developed and taught his first PBL-grounded literature course in an effort to stimulate his students to become active, critical readers. The course seemed both exciting and successful (See Sommers, 2002). However, that opinion was rooted only in anecdotal experiences. There really was no way to demonstrate what the students had learned – no small problem. But we subscribe to the redefinition of “problem” offered by Randy Bass. In his influential article, Bass argues that viewing a “problem” in teaching as an opportunity for research rather than as a failure to be overcome can transform a shortcoming into an ongoing intellectual activity. Bass notes that “As with scholarship or research, you cannot investigate everything at once. Indeed it may be that you can’t investigate more than one question at a time. What matters most is for teachers to investigate the problems that matter most to them” (1999, 8).
The problem that mattered was that we had no way to talk about the learning which had taken place in that exhilarating literature course. Salvatori and Donahue have commented about research in the field of English: “We have argued that a scholarship of teaching suggest more than conversations about the classroom and descriptions of instructional methods. Rather, it is the result of critical analysis, theorized reflection, and thoughtful enactment” (2002, 71). The desire to help students through their apprenticeship as literary readers led to the PBL approach. But the question of what happened to students as they experienced that apprenticeship was no more readily apparent in the new PBL course than it had been in earlier literature courses. Our need to learn more about what had transpired in the PBL literature course led to this study, designed to analyze and reflect on, using Salvatori and Donahue’s words, the learning outcomes, in the form of critical reading responses, of students in a second PBL literature course.
Background of the PBL Course
English 124 is entitled “Introduction to Fiction.” The course description explains that the course will focus on the elements of the fiction writer’s craft: plot, setting, point of view, etc. The course was designed as an introduction to historical fiction with the central problem posed for the students being the following question, the very same one that motivated Professor Ehrman to respond to The DaVinci Code:
When reading a book set in the past, how are readers supposed to know what to trust or believe, especially when on some occasions they
encounter actual persons, places, events from history and on other occasions are reading about cultures with which they are unfamiliar?
The problem presented by the course required students to explore possible answers to this very real, but abstract and messy question. PBL groups were required to select the novels to be read in the course and make a presentation on the selected novel, illustrating the reading strategies the group had been developing. In order to select an appropriate novel, the groups had to research literary criticism of the novels available (I provided a list of a dozen recent historical novels) and often delved into interviews with the authors as well as investigations of the historical events depicted. Later in the course, the groups had to develop a second presentation that illustrated their strategies for reading historical fiction, this time focused on one of two novels I had already selected because of the unique issues each one raised. Once again, the groups had to devise a plan for investigating the critical response to their chosen books as well as other ancillary materials that provided a context against which to read the novel. Finally, each individual student was asked to select a novel from those left on the short list I had provided, a novel about which they would write a take-home final exam in which the students would have the opportunity to apply what they had learned about critical reading to a text that presented the same reading problem with which they had been grappling for the entire term. (See Appendix B for course materials and a description of some of the PBL activities.)
The syllabus for English 124 reminded students of the university’s liberal education objectives for general education courses. Those objectives include
- critical thinking
- understanding contexts
- engaging with other learners
- reflecting/acting
This research project focuses on the middle two of those objectives. The course syllabus explained their relevance to English 124 as follows:
Understanding contexts complements critical thinking in that the ability to inquire freely requires our breaking through the boundaries of our personal understanding of the world, our personal contexts. All of the fiction we will read was written by writers who lived in the context of a particular time and place and whose way of thinking about and writing about the world was influenced by that context. Reading historical fiction demands that we enter another time and place – the past. If we are going to produce intelligent interpretations of our reading, we will have to be aware of the contexts in which they were created. But we too live in a particular time and place and our ways of thinking have also been influenced by our own contexts, so we will have to take into account who we are and where we come from as we read and interpret.
Reading may seem to be a solitary act, but that’s not entirely true. Of course, every reader encounters at least one other person during the act of reading – the author. When we read in a college course, we also encounter other students and a teacher; we are part of a reading community. And that is an enriching experience because each of us will learn regularly that other viewpoints and contexts exist when we listen to and talk with other members of our community. The class will emphasize not only individual reading activities but also meaningful small group interactions; among our activities, the class will be deciding upon the reading list for the course, for example. Engaging with other learners will be an integral part of English 124.
The syllabus also described more specific objectives focused on the specific PBL course:
- I want you to interrogate literary texts by asking questions about the choices the writer has made – choices about what to include, what to omit, what to
emphasize, what to de-emphasize, how these choices impact readers – and why the writer might have made these choices, and I want you to be able to
evaluate the effectiveness of those choices.
- I want you to become more adept at raising good questions about the “trustworthiness” or accuracy of the history you are reading; I want you to be able to
examine how the degree of “truth” you find in the text affects you and to be able to ask questions and speculate about how the writer has constructed that
truth with certain effects in mind.
As we pointed out at the outset of this essay, many literature courses involve students in critical reading, class discussion, and group work. This course, however, used a PBL framework as the foundation for those activities, not a typical strategy in literary study. All of the PBL activities were designed to prepare individual students to demonstrate their critical reading response skills on the final exam. The two instances described sketch a portrait of how the course used PBL to move students toward the final requirement of completing a take-home examination in which they were asked to demonstrate the reading strategies they had developed as solutions to the reading problems created by historical fiction.
A Map of Literary Reading Responses
In order to measure the students’ learning outcomes in the course, it was necessary to examine the degree to which their critical reading responses demonstrated achievement of the stated objectives described in the syllabus. With the consultation of a number of colleagues, we developed a taxonomy of critical reading responses, a rubric entitled A Map of Literary Reading Responses (see Appendix A).
Thomas Foster, in his popular text How to Read Literature Like a Professor, offers a preface in which he justifies his book by describing the difference between literary or critical reading – what we teach at the university – and what he terms “lay” reading: “When lay readers encounter a fictive text, they focus, as they should, on the story and the characters: who are these people, what are they doing and what wonderful or terrible things are happening to them? Such readers respond first of all, and sometimes only, to their reading on an emotional level; the work affects them, producing joy or revulsion, laughter or tears, anxiety or elation. In other words, they are emotionally and instinctively involved in the work. This is the response level that virtually every writer who has ever set pen to paper... has hoped for... When an English professor reads, on the other hand, he will accept the affective response level of the story...but a lot of his attention will be engaged by other elements of the novel. Where did that effect come from? Whom does this character resemble? Where have I see this situation before? Didn’t Dante (or Chaucer, or Merle Haggard) say that?” (2003, xv). Foster’s take on the task of critical reading, notwithstanding its breezy informality, articulates perhaps the main objective of reading literature in a college-level course: becoming a critical reader (Remember that Professor Ehrman’s objections to The DaVinci Code are grounded in the assumption that readers need to assess critically the historical accuracy of a fictional text. The professor rejects the interviewer’s suggestion that “‘It’s a novel, after all’” because luxuriating in the exhilaration of the plot and characters of the thriller isn’t the way he plans to read the novel.)
The Map of Literary Reading Responses reflects the distinctions that Foster makes. The Map’s three circles – initial responses to a text, intratextual responses to a text, and extratextual responses to a text – trace an incremental building of responses. They do not represent a hierarchy so much as they show how readers build on their initial responses, the ones Foster attributes to lay readers, and construct more sophisticated and critical forms of response. The Map presents this relationship among the three kinds of response by embedding them within one another as concentric circles.
The middle circle on the Map captures reading responses that focus intently on the words on the page as the readers try to construct meaning. These behaviors show a reader who is engaged in close reading, who is making active meaning of the text being read, and who is at times evaluating the craft of and the choices made by the author. These responses show evidence of critical reading of the sort most of the literature professors we consulted hope to promote in their courses. This form of response builds upon the initial responses and lays a foundation for another kind of response that reaches outside the boundaries of the text itself.
The largest circle on the Map describes readers’ attempt to interact with other readers and texts as they approach the reading act. These behaviors show a reader who, as part of an active reading process, has gone outside the text being read to learn more about that text. This active reader engages with the interpretations of other readers, the observations of the author, the history of the text’s composition, and/or the context of the historical setting of the text, and reevaluates the text in light of these additional materials. (Thus, relying upon Sparknotes or Cliff Notes to supplant the reading of the text, for example, would not qualify because such use is not a “critical” response.) These responses show evidence of critical reading of the sort most of the literature professors we consulted hope to promote in their courses. This form of response builds upon and works along with an intratextual response to the novel.
In her own literature courses, researcher Deborah A. Sarbin explains, she concentrates on students’ “over-reading,” which she defines as “moments when students offer an interpretation that interjects some material not in the text, for example, or that stretches logical credibility.” Sarbin is right that over-reading should be applauded. She concludes, “For students, I argue, over-reading is not simply mistaken reading, but rather an overreaching, a moment of trying too hard to be ‘scholars of literature’ rather than readers” (2003, 1). Sarbin’s goal is to help the students expand their repertoire of reading response behaviors into the middle and largest circles on the Map. While the English teachers whose commentary helped flesh out the Map of Literary Reading Responses were not in universal agreement about how to express these different responses, they were consistent in agreeing that the middle and largest circle of response behaviors represent preferred responses, as implied by Sarbin’s comments. They might not rank each form of Intratextual or Extratextual Response as equally important, but they would prefer to see their students engaged in more of these responses rather than responding solely with Initial Responses to a Text. This research study hoped to measure the degree to which students in the course engaged in those preferred reading responses.
Domains of Response and Course Objectives
The purpose of this research study was to describe learning outcomes, and that necessarily meant the focus would be on the behaviors in the middle and largest circles on the Map. However, only six of those ten domains were likely outcomes in the course, given the stated course objectives. The six domains studied were as follows:
- Constructing the text (developing interpretations beyond the literal meaning of the text’s actions; reading between the lines)
- Citing text (offering evidence to support interpretations/emotional responses)
- Recognizing textual moves (observing/assessing authorial choices/strategies in terms of the elements of fiction, e.g. plot, style, theme, etc.)
- Developing interpretive authority/exercising agency (expressing confidence in the value of personal interpretations through argument making; testing their own responses against the text)
- Exploring contexts (speculating about the cultural/biographical/historical backgrounds that influenced the text and/or its interpretation)
- Making intertextual connections (placing the text within the context of other readings and other texts)
These response behaviors, it is arguable, are gestures toward meeting the objectives of the university’s general education critical thinking expectations. The first two chosen domains – constructing the text and citing the text – seem implied in all of the course objectives as they constitute the very basic behavior of interpreting a text through close reading and offering evidence of that close reading. Critic Stanley Fish is talking about interpreting through close reading when he argues that poems–and by extension all literature – are “constructed artifacts, the products and not the producers of interpretation” (1980, 273). Fish’s observation presents the major dispute in literary theory over the past three decades: where does meaning inhere – in the text itself or in the response of the reader? However, his comment also voices a major given in literary studies: the text is a constructed artifact, and whether theory holds that readers construct the meaning or attempt to extract it from the text, in both cases readers are engaged in interpretation. For a definition of interpretation, we can turn to Robert Scholes, who says that readers “move to interpretation” when they consider the significance of what they have already noticed and begin to seek its meaning (1998, 117). Scholes aptly uses the verb moves to describe the readers’ efforts to engage in what we have termed “intratextual responses.” This movement is the students’ efforts to leave behind the inexperience that has made them, as Umberto Eco would say, “naive readers,” readers who expect unambiguous meanings to arise from texts (1979, 7-23). Readers leave that naiveté behind as they learn the roles they must play in constructing meaning from a text.
The syllabus also explained that students were expected to “interrogate literary texts by asking questions about the choices the writer has made,” an objective represented by the domain of “recognizing textual moves.” Scholes makes a case for the importance of this kind of response when he defines “good reading” as “reading every text sympathetically, trying to get inside it, to understand the intentionality behind its composition” (1998, 118). Scholes argues that critical reading is reading that notices and perhaps even evaluates what actually appears in the text. Even if literary critics may disagree about authorial intention (and they definitely do!), they can agree that critical readers still encounter the evidence of authorial choices in the very words and sentences on the page and can appraise the impact of those choices.
The syllabus also expressed an objective that the students would “become more adept at raising good questions about the ‘trustworthiness of the history” they were reading, an objective reflected by the domain of “developing interpretive authority” or argument making. “Those who hail the indeterminacy of all ‘texts’ are ...quite right,” says Wayne C. Booth, “up to a point: readers must always in a sense decide whether to accept a given responsibility” (1990, 354). Making decisions about their own responsibilities as readers is reflected in the course objective about evaluating historical trustworthiness; we tried to capture that assuming of responsibility in the notion of argument-making as evidence of a developing authority.
The syllabus also informed students that the course would help them in “understanding contexts” and in “engaging with other learners.” Behavior domains such as “exploring contexts” and “making intertextual connections” emphasize these same attributes of critical reading. The importance of exploring contexts as a literary response can be illustrated with an example offered by Annette Kolodny. She offers an argument that women writers may have been underappreciated in the past because male readers weren’t familiar enough with the worlds of these writers to read them appropriately. “The reader coming upon such fiction with knowledge of neither its informing literary traditions nor its real-world contexts will find himself hard pressed, though he may recognize the words on the page, to competently decipher its intended meanings” (1980, 305-6). This inability to “decipher” meanings suggests that a convincing argument can be made for examining the context of a literary work. Whether the context consists of biographical background, cultural studies, historical exegesis, etc. depends on the literary critic, but context matters to literature teachers. As for the importance of intertextuality, Theresa Rogers’ argument is that the “richest” readings of a text will include references to a number of other texts. She explains, “For instance, a reader who focuses solely on the story structure will likely produce a more limited interpretation than a reader who simultaneously focuses on a character’s reaction, comparisons to characters in another text, and his or her own personal responses (or texts)” (1991, 393). And while the reasons for making links to other texts and the kinds of texts chosen will differ from teacher to teacher depending on the theoretical construct the teacher chooses, literature teachers are, of course, always interested in readings that are not “more limited.”
The point of this overview has been to establish the appropriateness of the reading behavior domains we chose to include in the study. They are connected to the stated objectives of the course in the syllabus, and they corroborate the values of many literature teachers, even though those teachers embrace a variety of critical approaches to reading literature. The Map of Literary Reading Responses represents both the desired outcomes of the specific course being studied and the more general preferred outcomes that literature teachers across a variety of courses would welcome if demonstrated by their students.
Methodology
Our research project focused its analysis on the final examination in the course. This assignment gave the students a week to write a 5 - 7 page essay at home. The exam counted for 25% of the course grade, providing students with ample incentive to produce their best work, given that the outcome could raise or lower their final grade by almost two letter grades. The task on the final differed from a more traditional exam about a teacher-selected book. The examination question asked students to answer the course question, the problem they had been exploring all semester long (“When reading a book set in the past, how are readers supposed to know what to trust or believe, especially when on some occasions they encounter actual persons, places, events from history and on other occasions are reading about cultures with which they are unfamiliar?”) as they would apply it to the historical novel each of them had selected to read as an independent reading assignment outside of class. The independently-read novel was a requirement of the PBL approach, and students were encouraged to select a novel that they felt would assist them in grappling with the course’s central problem. Thus, not only did students have incentive to do their best on the exam, but they also had personal responsibility for choosing the subject matter of the exam. It seems reasonable to expect that this increased investment in designing the exam would serve as a further incentive to deliver an excellent performance. In other words, we had strong reasons to anticipate that the students would try their very best to succeed on the final exam.
In order to analyze the students’ performance in their final examination essays across the six chosen domains, we used a coding system that would identify the degree of accomplishment achieved by the students. We employed a four-point scale as follows:
Level 0= |
no instance of the behavior |
Level 1 = |
an instance of the behavior but in a superficial or undeveloped manner |
Level 2 = |
an instance of the behavior that formed the basis for developing a substantial argument or interpretation of the student’s novel, with substantial meaning sustained for a full-length paragraph or more or referred to more than once as a touchstone for the developing argument |
Level 3 = |
more than one instance of level 2 behavior |
Each examination essay was read and coded by two members of the research team. We read and coded an exam individually and then met to compare notes. Where we were not already in agreement, we discussed and negotiated until we came to an agreement.
Using this system, we were able to generate seven scores for each student: 0 - 3 in each of the six domains and an aggregate score that reflected overall performance in a range from 0 - 18. We then aggregated the students in a number of ways: by gender, by the degree of focus in the examination on the course problem, by number of previous literature courses taken, and by academic year at the university ( The study focused on ten students in a class of 19. These ten were willing to sign informed consent for their materials to be analyzed. In terms of course performance, it is fair to say that these ten students in the aggregate were more successful students than those who did not grant permission, based on average final course grade and average final exam grade).
Data and Analysis
Our results will allow us to claim that certain reading response behaviors were manifested by the students in the course, a course that emphasized a PBL approach. In other words, we can describe what is happening in the course in terms of critical reading behavior. Had those behaviors been noticeably absent, such results might suggest that PBL as an approach to literature is a pedagogy of limited utility. However, the results are promising in that they establish that an encouraging level of critical reading response did occur.
We worked with ten student examinations, each generating a score for six behavior domains. Analysis of the six domains shows that critical reading response behaviors occurred to varying extents, as one would expect (see Table 1).

We interpret results at levels 2 and 3 to be indications of significant critical reading behavior and results at levels 0 and 1 to be indications of no significant critical reading behavior. A brief discussion of each follows:
- Constructing the text (developing interpretations beyond the literal meaning of the text’s actions; “reading between the lines”): A majority of the students (7)
demonstrated this behavior, as one would hope in a literature course. Given the examination question, it was possible to make a convincing argument without
resorting entirely to a close reading of the text, but the exams nevertheless show a substantial amount of interpretive reading of the students’ selected novels.
- Citing text (offering evidence to support interpretation/emotional responses): The results indicated that almost all of the students cited their novel’s text in their
exams; however, the level of development of this skill was evenly balanced between those who did so substantially or those who did not demonstrate the
behavior or only did so superficially. This is a disappointing result.
- Recognizing textual moves (observing/assessing authorial choices/strategies in terms of the elements of fiction, e.g. plot, style, theme, etc.): Fully 90% of the
exams received ratings at the highest level in this category, an understandable result given the course description which emphasized this reading behavior.
Nonetheless, this result is encouraging because of the students’ level of achievement; while they may have engaged in this form of reading response because it
seemed to be required, they did so with skill.
- Developing interpretive authority/exercising agency (expressing confidence in the value of personal interpretations through argument making; testing their own
responses against the text): The exam asked students to provide an answer to the course question, the focus of the PBL learning. In doing so, we interpreted
argument-making about the genre of historical fiction as an example of this reading response behavior. In other words, when a student offered an interpretation of
the novel, she evidenced “Constructing the text,” as described above. When the student offered a generalization about reading historical fiction based on the
reading of the chosen novel, she evidenced this reading response behavior. We found that 60% of the exams succeeded at the highest level in this type of
response. This behavior most closely relates to the PBL work performed in leading class discussion and presenting to the class. Given the demands of the
exam, it is not surprising to find all the exams show evidence of this behavior, but the level of accomplishment is heartening.
- Exploring contexts (speculating about the cultural/biographical/historical backgrounds that influence the text and/or its interpretation): This is one of the two
behaviors from the Extratextual Critical Response to a Text level of the rubric. It is a behavior that was emphasized throughout the semester in the course
because of the nature of the problem students were asked to tackle, a problem that was conducive to the use of extratextual materials. The exams showed a
majority of the students working at a substantial level in this domain (N = 6); the single largest group of scores was at Level 3 (N = 4). In our personal
experience, students in introductory level literature courses do not customarily consult extratextual materials, so this was a gratifying result.
- Making intertextual connections (placing the text within the context of other reading and other texts): These results were surprising and disappointing. All of the
course texts were historical novels, and the students themselves had responsibility for picking many of them. We had expected the exams to make extensive
cross-references to the other novels. It was entirely possible to produce an excellent exam without reference to other fiction, but 40% of the exams made no
references at all and only 30% demonstrated this behavior in substantial ways.
We also aggregated these results in several different ways to see whether different groups of students performed differently on the exam. Comparing the performances of female and male students (see Table 2), a statistical analysis determined that no significant differences in performance occurred, although it is interesting to note that a majority of the women cited text substantially in their exams while a majority of men did not. (We thank Dr. Samuel R. Sommers, Assistant Professor of Psychology, Tufts University, for serving as our statistical consultant.)

When we divided students into three groupings based on their previous literature course experience – 0 previous courses, 1 previous course, 2 or more previous literature courses – again no statistically significant differences in performance were discernible (see Table 3).

Most interesting, however, are the results when we grouped the students by their academic year as first-year students, second-year students, or third/fourth year students (see Table 4). While most of the behavior domains did not demonstrate statistically significant differences in performance, in Developing Interpretive Agency (p = .053) and Exploring Contexts (p = .066), there were significant differences in performance across the groups; more experienced students outperformed less experienced students.

What makes this noteworthy, however, is one additional measure we used. We came up with an overall performance score for each exam by adding the levels in all six behavior domains (0 - 18). Using these overall scores (see Table 5), we discovered no significant performance differences when the exams were grouped by gender and number of previous literature courses. However, when we grouped the students by their academic year, we found that the more advanced academic status held by the student, the better his or her performance (p = .001): First-year students averaged 5.5, second-year students averaged 11.8, and third/fourth year students averaged 16.0. Coupled with statistically significant differences in performance in two of the six behavior domains, academic year appears to be a factor in student critical reading performance.

Another way to examine the results is to determine the percentage of scores at each of the four code levels. Close to half of the total coding scores we issued were at the highest level, level 3, and the combined total of level 2 and level 3 scores exceeded 60% of the total scores (see Table 6).

When we contrasted the performances based on gender (see Table 7), the findings were interesting: 71% of the women’s coded scores were at the highest levels (Levels 2 and 3) while 56% of the men’s were at those levels. However, slightly more women received scores at the 0 level (17% vs. 15%) while almost one-third of the men’s scores were Level 1, meaning a superficial form of the reading response. A statistical analysis, however, showed no statistically significant performance between men and women in any of the six domains nor on their overall performance, as mentioned earlier.

The results by number of previous literature courses (see Table 8) show that students with no previous literature courses experience actually did better (67% of their scores were levels 2 and 3 compared to 56% for students with one previous literature course and 61% for students with two or more previous literature courses), but these differences did not rise to the statistically significant.

In the other grouping by academic year, however, the differences were statistically significant. The overall results by academic year (see Table 9) show that the most advanced students, third/fourth year students, registered 92% of their scores at levels 2 and 3, compared to 64% for second-year students and 25% for first-year students.

It is important to acknowledge that the study was focused on a small number of final exam essays. Thus dividing the essays into smaller groups often resulted in exceptionally small groups. Despite the small sample, the trend of the data is encouraging. The primary finding of this study, it should be emphasized, is that desirable critical reading response behaviors occurred in the course. That means that the course can be defined as a success and provides more than mere anecdote to support the claim that critical reading can take place in a PBL-based literature course. Clearly, there is room for growth in certain forms of response such as making intertextual connections and newer students to the university may perhaps require more teacher intervention in order to achieve better outcomes, but the overall results paint a favorable picture of student achievement in the course.
Implications
For student learning
The results of this study make visible the substantial learning in terms of critical reading that students demonstrated in a PBL-based literature course. While the study did not attempt to establish a cause-and-effect relationship between the pedagogy and the learning outcomes, it does establish that such learning has occurred in a nontraditionally taught literature course. Some inferences therefore may be drawn from the study:
- It would be productive to explore several intellectual development schemas (e.g. Perry, Belenky, Magolda-Baxter) and see how PBL pedagogy, student learning
outcomes, and intellectual developmental stages are related. Our research suggests that more intellectually advanced students perform better in PBL courses,
not surprisingly, but the main issue is whether PBL of this sort presents a realistic opportunity for all students to succeed. Further study is needed.
- Student learning took place in significant ways that most literature teachers would find gratifying. Even though it cannot be established that the PBL approach
led to these results, there are some strong hints that that may be the case: final exams which engaged fully with the course PBL focus were judged to show
more substantial critical reading behaviors than exams which did not engage with the course PBL focus; additionally, the exams were focused on novels chosen
by the students as part of the PBL process.
- Given that the population participating in the study consisted primarily of the stronger students in the course, it seems appropriate to conclude that the results
illustrate in some ways a “best case scenario.” What happened across the board in the course is beyond the scope of the study, but for those who succeeded,
as measured by the quality of their exams and the quality of their final grades, the course stimulated significant critical reading responses.
For teaching
The idea of using a problem around which to organize a literature course continues to hold promise. The study shows the kind of critical reading that took place in the course met the objectives set for the course. However, the low rate of enrollment in the study is problematic. Maryellen Weimer writes: “The gap between the one who knows (the teacher) and those who don’t (the students) is much larger than I ever realized. Sometimes you literally cannot figure out what you need to ask. And then when you finally do get a question framed, sometimes the one who knows doesn’t understand the question, probably because it’s framed in a way totally unrelated to what you do know and understand...learning takes courage. Teachers must respect that” (2003, 1, 3).
In future iterations of PBL-based literature courses, it seems essential to find strategies to bridge the gap that Weimer identifies between teacher and student. The knowledge at stake in a literature course is the experience of being a critical reader. Craig Nelson and Robert Grossman’s work on critical thinking and scaffolding seems appropriate here. For example, in English 124, the professor relied at times on author interviews, critical commentaries, author biographies, and contrasts with other assigned texts when he led class discussions of the reading. He intended his own approach to serve as a model for the students. To some extent, it appears that it did because the student groups also began using the same kinds of extratextual materials. However, their use was often superficial or confused.
The steps in between learning to find such material and learning how to apply it to a critical reading had been skipped, creating a gap that often frustrated both the students and the professor. And the students apparently did not see the value of contrasting a text with other texts they had already read. Scaffolding, by asking students to engage in activities making use of the extratextual materials the teacher provided, might begin to bridge that “gap” and encourage more students to take the courageous step of trying to learn to do something rather different and new for them. As a byproduct, our hope is that the cognitive dissonance would be reduced somewhat and the students would feel more confident, enough to be willing to allow their work to be studied so that a future study would capture a portrait of a more substantial group of students than the ten who participated in this study.
For research
We plan to continue exploring the data we have already collected. There are still other critical reading response domains on the Map that we can investigate, using the same population of exam essays. We would like to apply to the exams other critical thinking rubrics, such as the one developed at Washington State University, to see what information that would yield. We also would like to examine our data in light of student intellectual development theory (e.g. Perry, Belenky et al), as mentioned above, to learn more about how the observable learning might be connected to the students’ developmental level. Finally, we intend to replicate the study in another PBL-based literature course with a larger population of participating students to see if these results can be validated
Perhaps one of the major contributions of the study, however, rests in the design of the Map of Literary Reading Responses which can be employed as a rubric to assess any literature course, be it PBL or not. By tying the selected behaviors on the map to the stated course learning objectives, experienced readers of literary essays could assess the critical reading responses in other literature courses. The Map can readily be converted into a more traditional scoring rubric with a four-point scale and descriptors of each level of response.
Conclusion
Our study describes how students read critically in several desirable reading behavior domains. Courses that offer students the opportunity to think for themselves and to teach both themselves and the teacher can encourage students to read literature critically and actively, to become creators of interpretations rather than consumers. The results indicate that students with limited background in literary study learned successfully although there is some suggestion that more mature students in terms of their academic year might perform better. Thus, we are willing to assert rather cautiously that there is a place for PBL instruction in the literature curriculum. We are confident that the PBL approach has a place in the upper division literature curriculum and are encouraged to continue using PBL in introductory literature courses, despite the challenges it presents.
Works Cited
Bass, R. (1999, February). The scholarship of teaching: what's the problem? Inventio,1, from http://www.doit.gmu.edu/Archives/feb98/randybass.htm.
Blau. S. (2003) The literature workshop: teaching texts and their readers. Portsmouth, NH: Boynton-Cook.
Booth, W. (1990). Who is responsible in ethical criticism, and for what? The Company We Keep: An Ethics of Fiction. Berkeley: University of California Press, 1990. Reprinted in Richter, 349-355.
Cromwell, L. (2005). Reading and responding to literature. In T. Riordan and J. Roth (eds.), Disciplines as frameworks for student learning: teaching the practice of the disciplines(pp. 77-94). Sterling, VA: Stylus.
“The Dan Brown Code,” (2004, November 21). New York Times. Arts and Leisure, p. 2.
Eco, U. (1979). The role of the reader: explorations in the semiotics of texts. Bloomington: Indiana University Press.
Fish, S. (1980). How to recognize a poem when you see one. Is there a text in this class? Cambridge: Harvard U Press. Reprinted in Richter, 268-278.
Foster, T. (2003). How to read literature like a professor. New York: HarperCollins.
Hmelo-Silver, C. (2004, September). Problem-based learning: what and how do students learn? Educational Psychology Review, 16(3), 235-266.
Hutchings, B. and K. O’Rourke (2002). Problem-Based learning in literary studies. Arts and Humanities in Higher Education.1 (1) 73-83.
Hutchings, P. (2000). Approaching the scholarship of teaching and learning. In P. Hutchings (ed). Opening Lines: Approaches to the Scholarship of Teaching and Learning (pp. 1-08). Menlo Park, CA: Carnegie Foundation for the Advancement of Teaching.
Kintgen, E. (1983). The perception of poetry. Bloomington, IN: Indiana U Press.
Kolodny, A. (1980). Dancing through the minefield: some observations on the theory, practice, and politics of a feminist literary criticism. Feminist Studies, 6 (1), 1-25. Reprinted in Richter, 302-309.
Meyers, C. (1986).Teaching students to think critically. San Francisco: Jossey-Bass.
Nelson, C. and Grossman, R. (2004, May 10-12). Increasing the retention of under-represented groups–and the learning of all groups–in science, mathematics, engineering and technology courses. Chautauqua Short Courses for College Teachers, 2004 Development Program, University of Dayton.
Peskin, J. (1998). Constructing meaning when reading poetry: an expert-novice study. Cognition and Instructor. 16 (3), 235-263.
Richter, D. (2000). Falling into theory: conflicting views on reading literature. (2nd ed.) Boston: Bedford/St. Martin’s Press.
Rogers , T. (1991). Students as literary critics: the interpretive experiences, beliefs, and processes of ninth-grade students. Journal of Reading Behavior. 23 (4), 391-423.
Salvatori, M. and P. Donahue. (2002). English studies in the scholarship of teaching.. In M. Huber and S. Morreale (eds.) Disciplinary styles in the scholarship of teaching and learning: exploring common ground (pp. 69-86). Washington, DC: AAHE.
Sarbin, D. (2004 Spring). Re-inventing the university: student over-reading as apprentice over-reaching. Inventio , 6 (1), from http://www.doit.gmu.edu/inventio/past/display_past.asp?pID=spring03&sID=sarbin.
Scholes, R. (1998). A fortunate fall? The Rise and Fall of English. New Haven: Yale University Press. Reprinted in Richter, 111-119.
Sommers, J. (2002, November 21). Problem-based learning (pbl), multimedia, and contemporary american fiction. 22nd Lilly Conference on College Teaching, Oxford, Ohio.
Squire, J. (1964). The responses of adolescents while reading four short stories. Research Report No. 2. Champaign, IL: NCTE.
Weimer, M. (2003). Teaching professor learns lessons of being student. The Teaching Professor, 17 (3), 1 and 3.
Appendix A: Map of Literary Reading Responses
Back to Top
|