8 Central Reasoning Assessments

Most discussions of the question how (or whether) critical thinking can be assessed focus on the general skills that it requires: the ability to judge the evidence for and against a conclusion, the ability to identify assumptions, and so on. As important as this question is, a full account of critical thinking must also consider how it should manifest itself within specific disciplines and subject areas (history, English, biology, etc.).

In this chapter, I describe some interrelated ways to assess critical thinking in a discipline.^[1] I will describe one of these in detail, and the others I will just outline. Each assessment strategy focuses on reasoning through, and in terms of, the most central parts of the discipline or field in question, and might therefore be called a central reasoning assessment. Such assessments are holistic: they require both analysis and synthesis; they require students to frame, organize, and sometimes identify the problematic situation to be addressed, to bring to bear insights from the course as a whole, and to reason within the discipline. For a number of reasons, these assessments satisfy the underlying goals of teaching the discipline. (Arguably, they do this better than do assessments based on individuated skills.) I will therefore end the chapter with a discussion of the implications that central reasoning assessments have for teaching.

Fundamental and Powerful Concepts

The first assessment I want to describe involves fundamental and powerful concepts (I will use the abbreviation “f & p concepts” throughout this chapter). In a course in a discipline (and, I would argue, in a discipline itself) there is a small set of f & p concepts. By “f & p” concepts, I mean those concepts which, if I could get students to understand them deeply, would enable students to understand a great deal of the course. Examples might be the concepts of social patterns and social forces in sociology; romanticism in nineteenth-century Western literature, music, and art; place in geography; managing, marketing, and finance in business; audience in writing; and what is justifiable in ethics.

Fundamental and powerful concepts can be contrasted with concepts that have a narrower, more restricted application. Cell versus mitochondria is an example. Cell is a much more fundamental and powerful concept in biology (think of a general-education biology course) than mitochondria is. (Homeostasis is probably even more f & p). Students who achieve a good grasp of the concept cell will be able to think through and gain insight into a very large number of topics in biology. If they think those topics through using the concept cell (in a way that is clear, accurate, and relevant, and that identifies relevant assumptions, possible alternative explanations, etc.), they will be thinking critically in the discipline. In addition, a good grasp of the concept cell will help students to think critically about a huge range of topics they will encounter outside the course. By contrast, a student who achieves a good grasp of the concept mitochondria will not thereby gain insight into nearly as large a range of other biology topics.^[2]

F & p concepts are fundamental in the sense that they underlie — logically underlie — a large number of other concepts in a discipline. By explaining restricted, narrower concepts in terms of more fundamental ones, students grasp (or recreate) part of the logical structure of the discipline. F & p concepts are powerful in the sense that they illuminate a large number of problems and situations in a wide variety of settings. Thus, they are not simply concepts to be thought about. Rather, they can better be described as concepts-in-use: as tools that are useful for thinking about other things.

A General Template for Assessments in Terms of
Fundamental and Powerful Concepts

The basic form of a question using f & p concepts is as follows:

Explain the following problematic situation [problem, question, event, situation, state of affairs, fact, argument…]. Do that by using the fundamental and powerful concepts of the discipline as the key parts of your explanation. Give good reasons to back up your explanation.

In accordance with this general template, assessments in terms of f & p concepts ask students to reason their way through problems using the f & p concepts. In many contexts, the instructions can ask students to focus on situations, questions, states of affairs, events, arguments, almost anything in fact. In giving their explanations, students are required to give good reasons to explain why they analyzed the situation the way they did (though instructions to do so will not always be explicitly stated in the question itself, as I discuss below). It is important that the problem or situation to be explained is one that has not been explicated in this respect by the teacher or in the reading for the course. Otherwise, the question requires only that the student recall what the teacher or text has explained.

Variations in instructions

Assessments in terms of f & p concepts may be presented in a great variety of ways. This variation is one of the strengths of such assessment. Instructions for answering questions can vary according to:

• how specific and well-defined the question is;
• whether the problematic situation is to be identified by teacher or student;
• whether the appropriate f & p concepts are specified by the teacher or identified and then used by the student; and
• whether critical thinking dimensions are explicitly included in the question or must be seen as relevant and introduced into the response by the student.

The following list contains five variations in the kind of instructions that can be given to students. Each calls for the exercise of different sets of critical thinking skills. (The sample questions use general-education biology as the discipline and cell and homeostasis as the relevant f & p concepts.)

1. Questions that are specific, and well-defined, where both the problem or situation and the f & p concepts are specified by the teacher:

1. Reason out the following problem using the concept of cell and homeostasis.
2. Explain what is occurring in the following situation, and why it is occurring, using the concept of homeostasis as the key concept in your explanation.

2. Questions that are specific, and well-defined, where the problem or situation is identified by the teacher, but where the f & p concepts to be used are left unspecified by the teacher:

Reason out the following problem using the most appropriate f & p concepts.

3. Questions that are moderately well-defined:

Chapter 7 discusses “Deriving Energy from Food.” Explain the most important ideas in that chapter using the concepts cell and homeostasis.

4. Questions that are not well-defined:

Respond to the following situation as seems appropriate (or as seems appropriate biologically).

5. Questions in which the teacher specifies context, but the problem or situation is to be identified (and then framed and reasoned through) by the student:

Look around this room at this moment, identify a significant situation or state of affairs that is within the domain of the discipline. Explain it, using the most appropriate f & p concepts in the course. [Variants include “Look at this video, this novel, this essay, this newspaper, your family situation, the world today, your future as you envision it…”.]

One important assumption behind these assessment strategies is that the more parts of an answer that are supplied by students themselves (instead of being contained explicitly in the written instructions), the more authentic the assessment of critical thinking in the discipline (see the discussion that follows). For example, in #4 the instructions supply only minimal information, asking students to respond to a described situation “as seems appropriate,” or “as seems appropriate biologically.” Students will have to frame the problem themselves — decide what is appropriate to address. This is fitting because it is precisely what needs to be done by anyone who is thinking critically about a real situation in terms of biology.

The level of authentic assessment is increased in #5. The goal in that type of question is to do as little identification of the situation to-be-addressed as possible. The idea behind this strategy is that for people to think biologically about events they encounter in their lives outside the classroom, they must first be able to recognize and select (from the stream of their experience) those situations they need to think about in terms of the discipline.^[3] This crucial part of learning to think within a discipline is seldom assessed.^[4] It involves a disposition to see the discipline as relevant to one’s life beyond the school setting. For example, if my brother is diagnosed with cancer, and I am confused about what is happening to him, I have to recognize that I might gain clarity by thinking through the situation in terms of the concept cell.

In varying the level of authentic assessment, a teacher may also decide to include more specific instructions in the question. As a teacher, I might choose to add any of the following to the kinds of questions outlined in #1-5:

In your explanation, in terms of f & p concepts, you should exercise the following critical thinking skills:

• give good reasons to back up your explanation;
• identify two key assumptions you are making;
• identify at least one alternative explanation someone might give;
• identify the strongest objection someone might make to your explanation; and
• identify the part of your explanation that is most questionable.

The teacher need not refer to the above required elements of the explanation as “critical thinking” skills or abilities. But it is important to note that these more specific central assessment instructions highlight elements of critical thinking skills or tools as identified by many other authors in this volume (particularly Johnson, Sobocan, and Case).

Giving more specific instructions along these lines is particularly appropriate near the beginning of a course, when the goal is to teach students some of the elements that are necessary to address in thinking through almost any question in the discipline. In explicitly asking students to identify the key assumptions they are making, I teach them that the identification of assumptions is a crucial step if one’s response is to qualify as a critical thinking response. By later omitting those explicit instructions in questions of the sort outlined in #1-5, I am requiring students to identify assumptions without being prompted to do so. That is an important way to help students internalize the necessity of critical thinking in responses.

Additional variations

Another, more advanced, assessment alternative recognizes that f & p concepts in a discipline are not automatically adequate for explicating all ideas in that discipline.^[5] In view of this, an important critical thinking skill is deciding when (and to what extent) a problem or situation cannot be adequately explained in terms of f & p concepts.

6. A problem to be identified within a context is one that is (or seems) not adequately explainable via the f & p concepts.

1. Identify a significant topic from Chapter 4 that cannot be explained using the concepts cell and homeostasis.
2. Identify some situation that is within the domain of the discipline, but which you cannot adequately explain using the f & p concepts in the course.

1. What questions would you need to answer for the explanation to proceed?
2. What further information would you need for the explanation to proceed?
3. How would you research that information?

In this case, the sub-questions are possible add-ons that can be used to assess students’ skills in identifying relevant follow-up questions and in mapping out a plan of research. Both of these are major skills in learning to think critically within a discipline.

One virtue of assessment in terms of f & p concepts (and the other central reasoning assessments below) is its ability to infuse central critical thinking into standard assessment questions in a course — for example, those in the exercises at the end of a chapter in a textbook — whether those original questions required critical thinking or not.

1. Take a standard, specific critical thinking essay question and change it into one that requires deeper, more critical thinking in the discipline.

[original] “What ecological consequences would occur if humans, using a new and deadly fungicide, destroyed all fungi on earth?” (Audeskirk 1999, 389)^[6]

[add-on] After you have answered the original question, explain your response in terms of the concepts cell and homeostasis.

2. Take a standard, highly specific problem from the chapter exercises, one that involves recall of information or rote problem-solving skills, and transform it into one that requires deeper critical thinking within the discipline.

1. [original] “Diagram the internal structure of leaves. What structures regulate water loss and CO₂ absorption by a leaf?” (Audeskirk 1999, 479)
2. [original] “Oxygen is released to the atmosphere in the light-dependent reactions of photosynthesis when water is split to supply electrons to:

1. Photosystem I
2. Photosystem II
3. Calvin cycle
4. C₄ pathway
5. CAM” (Krogh 2000, 166)

[Add-on to a. and b.]: Explain your answer using the concepts of cell and homeostasis.

Note that questions of this sort can require any degree of discipline-based depth or precision that the teacher deems appropriate, so no loss of rigour is entailed by central reasoning assessments.

A similar virtue is that, by using f & p concepts, assessments can be constructed that require greater levels of synthesis. Very much of critical thinking testing has always emphasized analysis at the expense of synthesis, but with no real rationale other than that analysis is easier to test for. In fact, all central reasoning assessments require students to assemble and comprehend the broad, large-scale structure inherent in a discipline. Consider the following questions:

9. What follows is a list of 18 “key terms” from the end of Chapter 8. Organize the most important of those terms into a coherent overall scheme, using the concepts of cell and homeostasis as your foundation. Note any terms that do not fit within this organization and explain why they do not fit.
10. The end of Chapter 12 contains ten review questions and six multiple-choice questions. After answering them, explain how the questions are interrelated, using the concepts of cell and homeostasis.
11. Look at the Table of Contents of your biology textbook. Review the headings of the 16 chapters. Explain how those 16 headings form a coherent picture of life. Use the concepts of cell and homeostasis to organize your synthesis. Give reasons and alternative explanations when appropriate.

Scoring student responses

Student responses to central assessments can be scored using general elements and standards of critical thinking as well as those that are more discipline based. Thus, a response can be scored on:

• the extent to which it is clear and accurate;
• the extent to which it is backed up by reasons, evidence, and supporting details;
• whether it focuses on what is most important in responding to the problem (in contrast to listing unimportant information or details);
• whether it takes adequate account of complications that may arise;
• how comprehensive it is;
• whether relevant alternatives are addressed; and
• the extent to which it takes adequate account of the assumptions, interpretations, and inferences being used in the explanation.^[7]

Other Central Reasoning Assessments

Using f & p concepts allows one to construct assessments that focus on some central ways of thinking critically within a discipline. A number of other tools for assessing critical thinking in a discipline work in roughly the same way by:

• addressing the most central question of the course;
• seeing the world (i.e., interpreting situations) from the point of view of the discipline (including domain, categorizing of that domain, and connections among those categories); and
• analyzing the logic of the discipline as a whole.

The central question of a course is the question that underlies the course as a whole. (I speak of the central question, but there can be several — though not many.) The central question in an educational psychology course could be formulated in this way: “How do students learn and how can I help students learn?” In a biology course, it might be, “How do living things work?” or “How do living things get to be the way they are?” These kinds of overall questions often get lost in the abundance of details in a course — yet it is the question that shapes the entire course. To answer a central question in an essay requires students to organize a well-thought-out way of fitting the whole together and to bring to bear insights from the entire course.

Asking students to write a response to the central question of the course serves as an excellent pre- and post-test for learning to think critically within a discipline. The difference between student responses at the beginning and the end of the course should display not merely a greater amount of information, but a substantially different way of approaching, organizing, and reasoning through the central question. Again, it is essential in central reasoning assessments that the teacher not present his or her direct answer to the central question during the course — otherwise the students’ “well-reasoned responses” at the end may simply be mirroring the teacher’s answer.^[8]

Machine-Scorable Central Reasoning Assessments

Assessments in terms of f & p concepts, as described so far, are open-ended questions, requiring shorter or longer written responses. The same is true of the other central assessments mentioned above. Many of these questions, however, can be adapted to become machine scorable as multiple-choice or multiple-rating items. There are any number of ways to do this, the simplest being for teachers to construct various responses (either created themselves or taken from open-ended responses by students to previous assessments) to variations #1-11, and then to ask students to rate them according to critical thinking standards, such as those listed above for scoring student responses. The conversion from open-ended to machine-scorable items brings with it a significant loss of authenticity in assessment, but there are of course distinct gains as well, particularly the feasibility of assessing large numbers of students.

Assumptions about Learning: Implications for Teaching

Central reasoning assessments are far enough removed from standard practices of teaching and assessing that I want to address some of their salient features and the implications they have for teaching, particularly for the question of how one should structure the attempt to teach students to think critically in a discipline. These implications concern the shift in course focus brought about by f & p concepts, the increased responsibility placed on students, and the goals of teaching general-education courses.^[9]

Focus

Central reasoning assessments require students to attain greater mastery of a small number of flexible, widely applicable, discipline-central concepts and ideas rather than a cursory understanding of a much larger number of concepts that are only more or less central. An assumption I make in this chapter is that there is a distinct benefit in building a course around f & p concepts for learning to think critically in a discipline. F & p concepts are concepts-in-use: they are not simply to be learned about, but to be internalized and used to think about other things (lenses, rather than merely objects). They are organizers that help to put parts together into a coherent whole so that the students will not get lost in the details. F & p concepts constitute the most central, versatile, and widely transferable part of the discipline at the general-education level.

This approach clearly has implications for teaching. Teaching a course using dozens and dozens (maybe hundreds) of concepts (bold-faced terms in the text or in handouts, for example) is the most standard way of teaching and assessing in a discipline. In these cases, teachers sometimes take it for granted that students will get both an overview and the logic of the whole by studying the parts.^[10] For many students the emphasis on numerous narrow concepts promotes a scattered, disconnected, patternless way of seeing the discipline, with little awareness of what is central and what is peripheral.

Centering a course on f & p concepts can counteract this dis-connectedness. So, in a sociology course, students might study the same topics as before — culture, society, family, deviance, sexuality, and so forth — but now the course would be structured around the f & p concepts of social patterns and social forces. The focus of the course would be on having students take every important topic and learn to identify the social patterns inherent in it and then understand the social forces that bring it about. Assignments and projects would also require students to use those same f & p concepts to reason through situations they encounter in their lives outside the course.

Student responsibility

Central reasoning assessments place more responsibility on students, requiring them to organize concepts into a reasonable, synthesized hierarchy. In the usual format for classroom tests, by contrast, questions or problems (a) are usually defined carefully by the teacher or the text, (b) relate only to the chapter or unit currently being studied, and (c) concern concepts that appear largely disconnected from one another. None of these needs to hold true in a central reasoning assessment.

Clearly, this shift of responsibility also has teaching implications. The idea is to change the way the discipline is taught by focusing on helping students in three ways. First, the students must gain a strong grasp of the f & p concepts. Second, they must use the f & p concepts to link more specific concepts together into a logic. Third, they must then use those f & p concepts to gain insight — discipline based insight — into a larger range of problems (topics, situations, states of affairs, questions, points of view, etc.). The aim is for students to become more self-sufficient in thinking their way through a wealth of problematic situations using the same small stock of fundamental and powerful concepts. Teachers may well begin this process by taking well-defined problems and guiding students through the task of critically understanding them using the f & p concepts. As time goes by, teachers can play a less central role. They can begin to describe problems more sketchily, they can direct students to a setting (a case study, a video, a book, etc.), leaving the problem itself to be identified not by the teacher at all, but by the students. As the course progresses, the teacher’s role becomes more focused on providing guidance and feedback. The students will be required more and more to identify the relevant situations themselves, to frame them as problems to be thought through using f & p concepts, to organize their approach to understanding the situation in terms of the discipline, and then to carry out the explication themselves.

Goals of general-education courses

Central reasoning assessments facilitate the underlying goal of teaching a discipline as a general-education course. An assumption in this chapter is that such courses should be taught so as to benefit the students who are actually in them. One consequence is that non-majors and those who will not be professionals in the discipline should be taught knowledge and skills in the discipline that will benefit them as non-majors and non-professionals. That is the course should not be taught as if all students were majors when in fact they are not. Thus the underlying goal of a course, stated most generally, is to help students learn to use the discipline to identify, frame, and get insight into problems, questions, and situations that they will likely encounter in their lives, ones that will be important — important to them — to figure out.

Keeping central reasoning assessments in mind gives instructors a teaching guide, other than a textbook, by which they can select what is essential to achieving course goals. One such goal, perhaps the most important one (at least to those who contributed to this volume), is to emphasize the essential features of critical thinking throughout the course and in the evaluation of students’ understanding of a discipline. To help students attain thoughtful understanding instructors may well need to de-emphasize sub-topics, isolated skills, and details that are time consuming and less essential to the course. The skills targeted in central reasoning assessments are those that are essential both to thinking critically in a discipline within the classroom and to using discipline-based reasoning to enhance one’s life.

References

Audeskirk, T., and G. Audeskirk. 1999. Biology: Life on earth. Upper Saddle River, NJ: Prentice Hall.

Krogh, D. 2000. A guide to the natural world. Upper Saddle River, NJ: Prentice Hall.

Nosich, G. 2009. Learning to think things through: A guide to critical thinking across the curriculum. Upper Saddle River, NJ: Prentice Hall.

Paul, R., and L. Elder. 2001. Critical thinking: Tools for taking charge of your learning and your lift. Upper Saddle River, NJ: Prentice Hall.