PATRICIA M. KING
LEARNING TO MAKE REFLECTIVE JUDGMENTS
The reflective judgment model describes a sequence of changes in thinking that affects the ways students justify their beliefs and make judgments.
PATRICIA KING is an associate professor in the Department
of Higher Education and Student Affairs at Bowling Green State
University in Bowling Green, Ohio. She is one of the creators
of the reflective judgment model of adult intellectual development.
She and Karen Strohm Kitchener are completing a book on reflective
judgment theory, research, and educational applications; it will
be published in 1992 by Jossey-Bass. A portion of this article
is drawn from that book.
2 LIBERAL EDUCATION VOL. 78 NO. 1 JANUARY/FEBRUARY 1992
(Reprinted with permission.)
DISCOURSE
"How do we know? Why do we believe?
What is the evidence? Here, whatever the subject matter
we are at the heart of the intellectual process, concerned with
the phenomenon of humans
thinking, the processes whereby they establish a fact, put two
or more of them together, come to
conclusions as to their meaning, and perhaps even soar with some
leap of imagination to a
thought that has never been thought before."1
How do college students learn to answer these questions and reach meaningful conclusions in formed and convincing ways? What kinds of skills and understandings do they need to do so? In 1991 report, The Challenge of Connecting Learning, the Association of American Colleges examined the kinds of thinking skills ("habits of mind") associated with liberal learning. These include the ability "... to state why a question or argument is significant and for whom; what the difference is between developing and justifying a position and merely asserting one; and how to develop and provide warrants for their own interpretations and judgments."2
Clarifying educational goals such as these is a critical first
step toward teaching college students how to make thoughtful,
reasoned judgments. Such goals provide direction and clarity for
our educational efforts; without them, our efforts risk being
either badly off the mark or diluted by ambiguity.
How do students learn to think reflectively about complex problems?
Why does the assignment designed to teach reasoning skills excite
and energize some students but leave others cold or confused?
Why do students vary so widely in their interpretations of course
activities and assignments or in evaluations of teacher effectiveness?
These and other questions relating to the teaching of higher order
reasoning on the college level are explored in this issue of Liberal
Education. Each article includes an examination of students'
assumptions about knowledge and how it is gained as a foundation
for understanding how students learn to reason.
Students' assumptions are elucidated in the reflective judgment
model of adult intellectual development that Karen Strohm Kitchener
and I first described in 1981.3 This model provides
a foundation for understanding how college faculty and staff members
can responsibly assist students to question their assumptions
about knowing and learning and to make more reflective judgments.
No single model of intellectual development can capture and describe
adequately all of the complexities of human reasoning. Models,
however, can provide heuristic tools that teachers may use to
help understand some basic differences in the ways students reason
and make judgments. They also can help teachers learn how to take
these differences into account in encouraging students to think
more reflectively and make more reasoned judgments. We offer the
reflective judgment model here as one such heuristic tool.
The term "reflective thinking" was used by John Dewey
to describe the thinking process people use when faced with questions
of controversy or doubt for which their current understanding
or solution for whatever reason, no longer is satisfactory.4
According to Dewey, a "reflective judgment" is the end
goal of good thinking: the judgment or solution that brings
closure to the problem (if only temporarily).
While reflective and critical thinking can (and should!) be directed
toward a variety of types of problems, the reflective judgment
model specifically focuses on controversial problems where real
doubt exists about correct solutions (or best resolutions). Evaluating
the merits of alternative proposals to stimulate the economy,
weighing competing interpretations of national or international
political events, and deciding how to reduce pollution while respecting
economic interests are examples of what C. W. Churchman refers
to as "ill-structured" problems.5 Questions
such as these are full of the kinds of doubts to which Dewey referred
because they can be neither described with a high degree of completeness
nor solved with a high degree of certainty. Other types of intellectual
problems-such as calculating interest payments on a loan, converting
units of measure between metric and English standards, and balancing
a checkbook-can be described more completely and solved with higher
certainty; these are called "well-structured"
problems. The reflective judgment model describes a sequence of
changes in thinking that affects the ways students justify their
beliefs and make judgments about ill-structured problems.
The Reflective Judgment Model
For the last fifteen years, several colleagues and I have studied the ways people explain and justify their interpretations and judgments about controversial topics. We have found that individuals' answers to questions of what and how they know are related to their fundamental assumptions about knowledge itself and that these assumptions change during people's college years.
The reflective judgment model describes a developmental sequence
of increasingly complex and adequate ways of understanding and
resolving ill-structured problems. It demonstrates how a person's
basis of judgment (the way a belief is justified) is rooted in
his or her assumptions about knowledge itself.6 These
epistemic assumptions are implicit in individuals' decisions to
look for or ignore the facts of a situation, in the strategies
they use to gain information about a problem, in their
attempts to understand divergent interpretations, and in
the degree of certainty they feel about whether a problem
has been solved. This model attempts to make these assumptions
explicit and to show how they evolve over time.
In the steps toward reflective thinking described by the model,
people become better able to evaluate knowledge claims and to
explain and defend a point of view on a controversial issue. This
developmental progression in reasoning is described by seven distinct
sets of assumptions about knowledge and how it is acquired; each
set of assumptions, called a "stage," is associated
with a different strategy for solving ill-structured problems.
Following each of the descriptions below, a verbatim quotation
(printed in italics) from a Reflective Judgment Interview (RJI)-the
instrument we use to assess reflective judgment-is given to illustrate
the type of reasoning associated with these assumptions.
The early stages, 1 and 2, are characterized by the assumption
that knowledge is gained through direct, personal observation
or through the word of an authority figure and is therefore assumed
to be absolutely correct and certain: "I figure
if it's on the news, it's got to be true or they wouldn't put
it on." Thinking consistent with these assumptions
is considered "pre-reflective."
This level of assumed correctness and certainty is less apparent
in subsequent stages, but some vestiges remain. In Stage 3, for
example, absolute answers are assumed to exist but to be temporarily
inaccessible. "Right now. they are finding things about
the pyramids that can't be explained. Right now they are just
guessing.... Until there is evidence that people can give to convince
everybody one way or another, it's just a guess. Then it will
be knowledge." People who use these assumptions to guide
their reasoning tend to view all problems as though they were
defined with a high degree of certainty and completeness; as a
result, they are unable to differentiate between well- and ill-structured
problems. Moreover, until this absolutely convincing evidence
is in, no one can claim to "know" beyond his or her
own personal impressions: "It doesn't matter what you
believe or what I believe because until one of them finds an answer,
then I'm just going to believe what I want to believe. "
Stage 4 reasoning is less whimsical: Evidence emerges as an
important ingredient in the construction of knowledge claims,
along with the acknowledgment that the evidence itself cannot
be known with absolute certainty. Pragmatic reasons such as incorrect
calibration of measurement tools, the loss of data over time,
or the lack of a first-person account are used to explain this
lack of certainty: "I'd be more inclined to believe evolution
if they had proof. It's just like the pyramids. I don't think
we'll ever know [how they were built]. People will come up with
different interpretations because people will differ. Who are
you going to ask? No one was there."
At this level, differences between well- and ill-structured
problems are acknowledged (a developmental advance over the earlier
stages), but individuals often are at a loss when asked to make
judgments about ill-structured problems because they do not know
how to deal with the inherent ambiguity of such problems. Since
there are many possible answers to these questions, evidence that
can be used to support each answer, and no certain way to adjudicate
between or among answers, knowledge claims are viewed as being
idiosyncratic to the individual. Reasoning illustrated by these
assumptions is considered "quasi-reflective."
Stage 5 reasoning is characterized by a more complete, more balanced,
more detached analysis of the factors that contribute to a controversial
issue. Knowledge is seen to be strongly contextual, and any given
perspective is assumed to reflect its guiding principles, values,
or accepted rules of inquiry. Evidence and clear reasoning are
assumed to be necessary but not sufficient for constructing or
evaluating an argument, since it is further acknowledged at this
stage that evidence is not self-explanatory but must be interpreted.
"Since no two people are alike, they're thinking differently
and so they attack their problem differently. When they do that,
they throw out different material and so come from two different
positions since their intellectual curiosity is geared to different
things."
While these assumptions are useful for evaluating the adequacy
of arguments within a given perspective, individuals at this stage
are at a loss when asked to evaluate competing explanations or
proposals-tasks often associated with resolving ill-structured
problems. "There are other theories [of how the pyramids
were built ]that could be as true as my own theory] but based
on different evidence. Everyone has their own idea of what truth
is, but for myself, this is my explanation. But other people have
their own ideas on it and that's okay. "
Stages 6 and 7 represent the most advanced sets of assumptions
in the reflective judgment model. At this level, knowledge is
understood in relationship to context and evidence, and some interpretations
may he judged as being in some way better than others. These stages
reflect the assumption that one's understanding of the world is
not "given" but must be actively constructed and interpreted
and that knowledge must be understood in relationship to the context
in which it was generated. An additional assump-tion made at this
level is that some interpreta-tions or knowledge claims may be
judged more plausible than others. Criteria that might be used
in making such evaluations include conceptual soundness, coherence,
degree of fit with the data, meaningfulness, usefulness, and parsimony.
"One judges arguments by looking at such things as, how
well thought out the positions are, at what level one chooses
to argue the position, what kinds of reasoning and evidence one
would use to support it, how it fits into the rest of one's world
view or rational explana-tion, how consistently one argues on
this topic com-pared with other topics." Individuals
who make judgments in this way are exhibiting good "reflective
thinking."
The development of reflective judgment
Unlike other models of reflective or critical thinking, the
reflective judgment model is grounded in the underlying assumptions
of the cognitive developmental perspective articulated by Piaget
and Kohlberg. As such, this model also reflects assumptions about
the process of learning to think and reason more effectively.
Because these assumptions hold special implications for teaching
reflective thinking-or reflective inquiry, a concept introduced
by Barry Kroll elsewhere in this Liberal Education-I want
to describe them briefly.
Assumption 1: Individuals actively attempt to interpret
and make sense of what they experience. Examining an interpretation
from the perspective of the individual who constructed it often
reveals an internal logic to the explanation. Cognitive developmental
theories attempt to describe patterns (for example, common themes
or assumptions) that can be seen in individuals' explanations.
In the reflective judgment model, the observed patterns correspond
to stages (sets of epistemic assumptions) described above. The
importance of engaging students in the process of making
meaningful interpretations is evident in the Integrity passage
that opens this article.
Assumption 2: People's ways of making meaning develop over
time. Just as young children solve problems using different
strategies than do older children, so young adults understand
the worlds and reason differently than older adults, including
college teachers. Earlier ways of making meaning mature over time,
providing the foundation for later ways of making meaning.
More mature structures, furthermore, are assumed to offer more
complex, complete, and adequate ways of interpreting experience;
some of these ways might include considering more pieces of information
at one time, evaluating contradictory pieces of information, or
critiquing one's own as well as others' ideas. Earlier ways of
thinking are revised when individuals realize these ways have
become inadequate, necessitating the creation of new ways of understanding.
Such changes in how one thinks-which can be tantamount to a change
in one's world view reflect no small intellectual accomplishment.
Some students describe this change as nothing less than a personal
metamorphosis: "I used to think that everything printed in
a book must be true; now I realize how naive I was!"
This individual was acknowledging that she now thinks in qualitatively
different terms about sources of knowledge and her own role in
evaluating the truth of given statements. This reevaluation of
"old ways of thinking" is presumably what the authors
of The Challenge of Connecting Learning had in mind when
they noted,"Faculty members must take seriously what students
believe about given subject and engage their prior knowledge so
that new learning restructures the old, complicating and correcting
it rather than merely living side by side with it." 7
Assumption 3: Interaction with the environment affects an individual's development (in this case, intellectual development or reflective thinking). That is, environmental factors such as the types of intellectual challenges offered, quality of feedback, and opportunities for practice without fear of failing or being penalized affect how well a person learns to reason to conclusions. Although the direction of cognitive changes-that is, toward greater complexity and adequacy-is predictable, the rate of change for any individual fluctuates, depending in part on the characteristics of the environment, including the perceived amount of stimulation and support. In Dewey's words,"We never educate directly, but indirectly by means of the environment. Whether we permit chance environments to do the work, or whether we design environments for the purpose makes a great difference."8
Assumption 4: An emerging assumption is that individuals do not function "in a stage." Rather, individuals tend to exhibit a range of responses across stages. How an individual performs at a given point in time-that is, what particular stage characteristics he or she is evidencing- will depend on a variety of factors: how well the person was feeling or concentrating at the time, whether the person was distracted by another concern, the difficulty of the task presented, and the type of feedback and support offered.9 Teachers who wish to assess students' developmental levels should not rely on single examples of students' reasoning. A given example may be indicative of the level at which a student typically reasons, but it may underestimate the level at which she or he could function in other circumstances or learning environments.
Developing reflective judgment in the college years
The Reflective Judgment Interview has been administered to
more than one thousand individuals, including high school students,
traditional- and nontraditional-aged college students, graduate
students, and other adults. The results of these studies show
a strong relationship among age, educational level, and individuals'
epistemic assumptions. High school seniors score higher (that
is, show more reflective thinking and hold assumptions consistent
with more advanced stages) than do high school freshmen, but both
groups overwhelmingly evidence pre-reflective thinking. College
seniors score higher than college students at earlier class levels,
but as a group their thinking is best characterized as "quasi-reflective."
Advanced graduate students show the greatest use of reflective
thinking, and they score higher than beginning graduate students.
Thus, the developmental progression across educational levels
and ages is quite clear.
At least one thousand traditional-aged college students have taken
the RJI. While the scores have differed across institutions, the
mean scores for freshmen samples have ranged from 3.23 to 4.16.
(Mean scores correspond to stages: a mean score of 3.0 indicates
that the average score for a given sample was stage 3.) By contrast,
means for senior samples have ranged from 3.56 to 5.02. This progression
in scores across class levels and its consistency across studies
provide encouraging evidence of the beneficial effects of college."
10
These scores are discouraging, however, in two respects. First,
the differences between classes are not dramatic: the average
RJI score across freshman samples is 3.68, while the average RJI
score across senior samples is 4.02. Second, while these scores
indicate that seniors are reasoning in a manner that is at least
quasi-reflective-for example, using evidence to support their
beliefs (a key distinction between stages 3 and 4)-they are not
demonstrating an understanding of the role of evidence in making
interpretations (stage 5) nor defensibly critiquing their own
judgments as being in some way better than or preferable to
alternative judgments (stage 7), which is a hallmark of reflective
thinking.
In light of this research, let us now reexamine the proposals
that the Association of American Colleges offers as the foundation
for educational reform. Consider this assertion: "By attending
to the knowledge claims of the major over time and by treating
increasingly complex matters from multiple points of view, students
discover that nothing is self-evident, that nothing is simply
'there,' that questions and answers are chosen and created-not
given-and that they always are framed by context; for that reason,
they always are contingent." 11
The assumptions inherent in this statement- that knowledge is
contextual and answers are contingent clearly are consistent with
stage 5 of the reflective judgment model. Yet that is a full stage
above where most college seniors that we tested scored. Consider
a second example: "Students cannot be allowed to be content
with the notion that issues may be addressed by any number of
equally valid formulations among which they cannot choose. They
must learn to discriminate by arguing, and they must realize that
arguments exist for the purpose of clarifying and making choices."12
Here again, the skills associated with reaming to discriminate
the valid from the invalid (or less valid) formulations are associated
with the upper stages of the reflective judgment model (stages
6 and 7). Teachers who attempt to teach about this process to
students who do not think reflectively, or teachers who attempt
to do so without attending to students' underlying epistemic assumptions,
probably will be very frustrated, and their students probably
will be dissatisfied as well.
Enhancing the development of reflective judgment
Although a central goal of many educational programs is for
students to be able to reason using the types of reasoning skills
associated with the higher stages of reflective judgment, it is
important to accept and respect people at whatever developmental
levels they may be experiencing. While the characteristics of
early stages are not consistent with true reflective thinking,
nevertheless they are reflections of students' assumptions about
knowledge and how knowledge is gained-assumptions that some students
genuinely use to solve controversial problems.These assumptions
provide building blocks for subsequent ways of understanding and
resolving problems. Therefore, teachers must show emotional acceptance
for all students whose reasoning they are challenging and understand
that a strong internal logic exists in the patterns of reasoning
at each stage (including the early stages, which often appear
illogical). This understanding may make the student more receptive
when the teacher points out the insufficiencies of earlier ways
of reasoning and more willing to take the kinds of intellectual
risks associated with abandoning old, comfortable ways of thinking.
Thinking, reasoning, and judging are thus at the heart of the
intellectual process, and learning to make reflective judgments
is associated with participation in college programs and activities.
Encouraging students to develop the habits of mind associated
with reflective thinking and teaching them the importance of these
skills are among a college's most important responsibilities.
In addressing this responsibility, however, we must remember that
many factors influence this process of learning to make reflective
judgments: from a student's established reasoning skills, to a
student's emotional readiness to defend a point of view, to the
educational values emphasized in the college, to the broader societal
norms and expectations for reasoned discourse about current events.
In other words, learning to think reflectively occurs within the
context of an intellectual community. This community ranges from
the immediate environment of a student's living group or a specific
class to the broader community of the college and its environs;
each community has its special opportunities and limitations
for teaching students to think reflectively.
Teaching students to think clearly and complexly, to present their
arguments coherently and persuasively, and to weigh competing
claims is a complex and difficult process. Learning to think reflectively,
as defined here, involves some fundamental changes in students'
basic assumptions about knowledge itself and about the process
of reaming. How can college faculty and staff members responsibly
assist students in examining and reevaluating their assumptions
to make more reflective judgments? As suggested by The Challenge
of Connecting Learning, "The role of faculty members
is to provide structures and languages... that enhance and challenge
students' capacities to frame issues, to test hypotheses and arguments
against evidence, and to address disputed claims."13
What kinds of teaching strategies and learning environments do
instructors use and create in order to fulfill this role?
In this issue of Liberal Education, five faculty members
from both large and small institutions and from a variety of disciplines
address these questions. They show how they have used the reflective
judgment model to create educational environments that are particularly
conducive to teaching students the habits and skills of making
reflective judgments. The first two articles, by Barry Kroll and
David Finster, show how these ideas have been applied in the context
of specific classes in English and chemistry, respectively. Paul
Haas discusses a range of issues that emerge when teaching reflective
thinking to a special subgroup of students: those in honors programs.
The last two articles are directed toward wider cross-sections
of the campus community. Katherine Nevins discusses a program
designed for all faculty members on one campus. Finally, Carney
Strange suggests opportunities for promoting reflective thinking
outside the classroom by faculty and student affairs staff members
alike.
We invite you to join in dialogue with us about teaching students
to think more reflectively in answering the questions, "How
do we know?" and "Why do we believe?"
1. Integrity in the College Curriculum (Washington Association
of American Colleges, 1985), 15.
2. The Challenge of Connecting Learning (Washington Association
of American Colleges, 1991), 14.
3. Karen S. Kitchener and Patricia M. King, "Reflective Judgment:
Concepts of Justification and Their Relationship to Age and Education,"
Journal of Applied Developmental Psychology 2 (1981): 89:116.
4. John Dewey, How We Think: A Restatement of the Relation
of Reflective Thinking to the Educative Process (Lexington,
Mass.: D. C. Heath and Company, 1933).
5. C. W. Churchman, The Design of Inquiring Systems: Basic
Concepts of Systems and Organizations (New York: Basic Books,
1971).
6. While several developmental theorists informed the initial
conceptualization of this model most notably William G. Perry,
Jr., in Forms of Intellectual and Ethical Development in the
College Years: A Scheme (New York Holt, Rinehart &
Winston, 1968) and J. Broughton in The Development of Natural
Epistemology in Adolescence and Early Adulthood (unpublished
doctoral dissertation, Harvard University, the stage descriptions
summarized here are derived from extensive sets of interviews
with many individuals of varied ages and educational levels.
7. The Challenge of Connecting Learning, 13 (emphasis added).
8. John Dewey, Democracy and Education (New York: Macmillan,
1916), 22.
9. For references on stage theory, please contact the author.
10. Ernest Pascarella and Patrick Terenzini, How College Affects
Students (San Francisco: Jossey-Bass, 1991).
11. The Challenge of Connecting Learning, 13.
12. Ibid., 14.
13. Ibid., 4-5.