Evaluating Critical Thinking Skills

The guidelines suggested below propose how critical thinking skills can be assessed “scientifically” in psychology courses and programs. The authors begin by noting something about psychology faculty that is true of faculty in many other disciplines, which makes this article relevant to a much larger audience. “The reluctance of psychologists to assess the critical thinking (CT) of their students seems particularly ironic given that so many endorse CT as an outcome…” (p. 5) Their goal then is to offer “practical guidelines for collecting high-quality LOA (learning outcome assessment) data that can provide a scientific basis for improving CT instruction.” (p. 5) The guidelines are relevant to individual courses as well as collections of courses that comprise degree programs. Most are relevant to courses or programs in many disciplines; others are easily made so.

Understand critical thinking as a multidimensional construct – In their discussion of critical thinking in psychology, these authors propose that critical thinking includes skills, dispositions, and metacognition. Critical thinking skills in psychology include argument analysis and evaluation, methodological reason, statistical reasoning, causal reasoning, and skills for focusing and clarifying questions. Dispositions refer to “the willingness to engage in effortful thinking and the tendency to be open- and fair-minded in evaluating claims, yet remain skeptical of unsubstantiated claims.” (p. 6) Metacognition means being aware of one’s thinking and in control of it.

A recent article in The Teaching Professor highlighted the variation in definitions for critical thinking. These authors point out that critical thinking is either thought of generically or as being discipline-specific. They cite research that critical thinking is probably a combination of both. As a multidimensional construct, it contains some general reasoning skills and some skills that are specific to the discipline. The point is that if you want to assess learning outcomes associated with critical thinking, you cannot do that well without understanding how critical thinking is defined in your discipline.

Select important goals, objectives, and outcomes for assessment – What critical thinking skills and knowledge should students be able to demonstrate as a result of being in a course or program? Some faculty have learning goals so general that they are all but impossible to assess. They need further specification. If the assessment is to be scientific, then the goals, objectives, and outcomes must translated into specific hypotheses—ones that can be tested.

Align assessment with instructional focus – “Measures for assessing the impact of instruction must be sensitive to the changes instruction is intended to produce.” (p. 7) If the measures are sensitive, then classroom assessment can be used to look at the techniques being used, compare their effectiveness with other techniques, and conclude which are better.

Take an authentic task-oriented approach to assessment – Taking an authentic task-oriented approach means using a performance to assess how well students are completing a task. In psychology, tasks requiring critical thinking include evaluating the quality of information from the Internet, analyzing and evaluating research literature, using psychological theory to analyze and evaluate behavior, and writing research and case reports, among others. Many of those tasks can be used to evaluate critical thinking in a variety of fields.

Use the best and most appropriate measures – Because critical thinking has multiple dimensions, multiple measures should be used to assess it. The authors point out that standardized tests of critical thinking (the Watson-Glaser Critical Thinking Appraisal and the Cornell Critical Thinking Test are the two examples referenced in this discussion) are “probably better measures of general CT skill.” (p. 9) In many cases, no standardized tests or measures assess the specific type of critical thinking or aspect of critical thinking being developed in a particular course. In situations like this, new instruments may need to be developed.

Conduct assessments that are sensitive to changes over time – “Simply testing seniors once in their capstone courses is not sufficient to infer changes over time because the levels of skill and knowledge of students entering the program are unknown.” (p. 9)

Assess frequently, embedding assessment and feedback into instruction – Students can be assessed too much, especially if the same instrument is being used. They become sensitized to those instruments. The authors recommend a formative approach that embeds assessment in instruction. In this case, the assessment provides the instructor useful feedback and helps students focus on their development of critical thinking. It offers them feedback that can be used to improve their critical thinking skills.

Interpret assessment results cautiously and apply the results appropriately –
The quality of the data collected must be considered before decisions to change a course or a program are made. Not considering the quality of the data and not carefully interpreting the results can result in changes that do not improve learning outcomes.

Reference: Bensley, D. A. and Murtagh, M. P. (2012). Guidelines for a scientific approach to critical thinking assessment. Teaching of Psychology, 39 (1), 5-16.

Reprinted from Assessing Critical Thinking Skills, The Teaching Professor, 26.3 (2012): 4.

Posted in Educational Assessment
Tagged with assessing critical thinking, critical thinking, critical thinking skills

What is the best way to assess critical thinking?

The most effective way to measure critical thinking is to use a validated critical thinking skills test to assess the skills used to solve problems and make decisions AND to use a critical thinking mindset measure to assess the level of the person's consistent internal motivation or willingness to use his or her critical thinking skills when it counts in decision making.

Costly mistakes and poor judgments might be the result of deficient thinking skills, but an equally likely cause is a mindset that predisposes the decision-maker to biased, hasty or superficial analysis of the situation at hand.

Having the skill to interpret a situation and correctly infer that a problem exists would not be sufficient if we then cannot analyze why the problem continues to exist and then explain and evaluate our attempts to solve the problem.  Interpretation, inference, analysis explanation and evaluation are all critical thinking skills. 

  • A valid and reliable assessment of an individual’s critical thinking skills must challenge the individual to demonstrate all of these skills over a range of difficult problem situations that is calibrated to match the level of their decision responsibility. 

Critical thinking mindset determines how disposed a person will be to work to achieve goals and to engage and resolve significant problems.

  • Do they have the integrity to honestly define problem situations, the disposition to take an organized approach to seeking out best possible solutions, the tolerance to listen to all points of view, the ability to reconsider when the evidence points to the need for a new appraisal?  These are only a few of the characteristics included in a valid and reliable assessment of critical thinking mindset.  

For a complete assessment of an individual’s thinking, it is essential to measure both skills and mindset.   After more than 30 years of assessing critical thinking students of all ages and in working adults from all types of workplaces, the researchers at Insight Assessment have learned that there is an  array of reasoning skills and mindset attributes that are predictive of success in both life and work. All are important for reliable reasoning and decision-making.

Insight Assessment test instruments for children and adults include skills and mindset tools that can be paired for the most comprehensive information about the strengths and weaknesses in thinking.  We also offer a variety of two part tests that measure both skills and mindset. Contact us to learn more.

One thought on “Evaluating Critical Thinking Skills

Leave a Reply

Your email address will not be published. Required fields are marked *