CHAPTER 16
e-Learning to Build Thinking Skills

CHAPTER SUMMARY

When you help staff build thinking skills, you enable the workforce to quickly adapt to changing conditions. For example, in the military, Chatham (2009) observes: “Today’s missions now require that we also train each soldier to be a little bit of a linguist, anthropologist, city manager, arbitrator, negotiator, engineer, contract specialist, ambassador, and a consummate bureaucrat within the Army system. As if that weren’t enough, each soldier must be ready instantly to shift into a shooting mode and then an hour later calmly negotiate with the brother-in-law of the man he shot” (p. 29). How many job roles in your organization rely on flexible problem-solving skills? From managerial skills to consultative sales and customer service, nearly all organizations incorporate job roles with multiple competencies that require thinking skills to achieve bottom-line performance goals.

In this chapter we draw upon a number of research studies and reviews to update the guidelines described in the previous edition. Specifically, evidence suggests that (1) critical thinking skills can be improved through training and (2) explicit training programs that incorporate authentic problems and learner dialog are the most effective instructional approach. Successful teaching of thinking skills requires explicit teaching of job-relevant skills, modeling and discussion based on authentic problems, and a focus on job-specific strategies.

What Are Thinking Skills?

A 2014 American Management Association survey concludes that there is a need for managers and analysts in the United States who can ask the right questions and use the results of data analysis effectively. Specific skills identified include analytic thinking, problem solving, drawing conclusions based on data analysis, communicating findings, and decision making. Desired outcomes of these skills are improving strategic workforce planning, creating more efficient and targeted marketing, and increasing sales, profitability, customer satisfaction, and productivity. The report recommends that training departments close the gap with solutions that merge a broad understanding of finance, operations, and marketing with statistical analysis, presentation skills, and a focus on problem solving.

In Chapter 1 we made a distinction between near and far transfer skills. Near transfer skills refer to tasks that are performed more or less the same way each time, such as logging into your email account or processing a routine customer order. In contrast, far transfer skills require the worker to adapt knowledge, skills, and experience to tasks that involve unique situations and uncertainty. In other words, far transfer skills require thinking skills. Forty-seven percent of respondents to the American Management Association business survey stress a need for training of analytical skills.

But what exactly do we mean by thinking skills? Thinking skills are cognitive processes and strategies for solving problems. A recent report from the National Research Council (Pellegrino & Hilton, 2012), Education for Life and Work, noted that one of the most important 21st century skills sought by employers is the ability to solve problems—which can be referred to as problem solving, creativity, innovation, critical thinking, analysis, reasoning, argumentation, interpretation, decision making, adaptive learning, or executive function. In Table 16.1. we summarize three types of thinking skills: creative thinking, critical thinking, and metacognition.

Table 16.1. Three Types of Thinking Skills.

Type Description Examples
Creative Thinking/Adaptive Expertise Generating novel and useful ideas; solving unfamiliar problems Design an e-learning course Create a marketing campaign Troubleshoot an unusual failure
Critical Thinking Application of skills related to interpretation, evaluation, and inference Evaluate validity of an Internet resource Identify illogical reasoning in an argument  
Metacognition Your mind’s operating system responsible for setting goals, monitoring progress, adjusting approaches Assess what you do and do not know Identify skills you are not learning Monitor learning progress

By creative thinking we refer to the skill of generating novel and useful ideas and the ability to solve non-routine workplace problems. Carbonell, Stalmeijer, Konings, Segers, and van Merriënboer (2014) refer to this skill as adaptive expertise—contrasting it with routine expertise that underlies high performance in job roles or problems familiar to the worker. Adaptive expertise is required to solve problems or perform tasks that are significantly different from those experienced either in a training or work setting.

By critical thinking we refer to the skill of evaluating novel ideas or problem solution methods. A research review from the National Research Council identifies three major competency clusters associated with critical thinking: cognitive processes and strategies, knowledge, and creativity (Pellegrino & Hilton, 2012). In Table 16.2 we summarize the skills associated with each of these clusters. As you review these skills you will see considerable overlap in terms of their deployment in reasoned thinking. For example, a team assignment to create a social media freedom of information policy statement could involve: (1) online research and evaluation of bias and credibility of multiple sources, (2) identification of several perspectives on freedom of information, (3) synthesis of multiple perspectives, (4) development of a policy statement with supporting rationale, (5) vetting and revision of the policy statement based on stake holder input, and (6) communication of the policy to various stakeholders through written and oral media. These activities require information literacy, analysis of credibility and bias in sources, interpretation, argumentation, decision making, innovation, and communication skills.

Table 16.2. Thinking Skills: Cognitive Competencies.

Competency Skills Example
Cognitive Processes and Strategies Critical thinking, problem solving, analysis, reasoning/argumentation, interpretation, decision making, adaptive learning Developing a reasoned position summarizing multiple perspectives on a topic such as application of free speech rights in social media, followed by a policy statement for a social media platform.
Knowledge Information literacy (research using evidence and recognizing bias in sources); information and communications technology literacy; oral and written communication; active listening Using technology to source and assess credibility of online resources, identify biased reasoning, and present a report that synthesizes findings.
Creativity Creativity, innovation, adaptive expertise Completing a task or solving a problem that is new and unfamiliar, such as designing a new marketing campaign or web page or developing an emergency action plan.

Adapted from Pellegrino and Hilton, 2012.

Metacognition is the superordinate thinking skill of managing one’s thinking process, which includes planning, monitoring, and adjusting one’s solution process. In Chapter 15, we defined metacognition as the skill that sets goals, plans an approach, monitors progress, and makes adjustments as needed. People with good metacognitive skills focus not only on the outcome of the task, but on the rationale or process behind the decisions made to achieve that outcome. When working in a team, the person with high metacognitive skills will be the one to say: “Wait—let’s stop and see if we are making progress. Will our individual efforts come together?” When working on a problem alone, the person might say: “I’m hitting some dead ends here. Where can I get some help?” When a mission or project is completed, that person will organize a debriefing session in which lessons learned are articulated and documented. In other words, the metacognitive worker or team is mindful of their problem-solving progress and products.

Generic Versus Domain-Specific Thinking Skills

Instructors may debate the extent to which thinking competencies are composed of generic skills, such as those listed in Table 16.2, or are domain-specific, such as guidelines for troubleshooting automotive electrical failures or protocols for patient diagnosis and treatment plans. The consensus among researchers is that thinking skills are always embedded within domain-specific tasks (Pellegrino & Hilton, 2012), and, therefore, we recommend teaching thinking skills within the context of job-related tasks. Because problem solving in the workplace requires domain-specific knowledge, the most promising solution may involve (1) generic skills adapted to specific job roles and/or (2) domain-specific skills derived from analysis of expert performance in specific job roles.

Can Thinking Skills Be Trained?

An important question for workforce management is: Can thinking skills be trained? If so, is a generic or domain-specific or combination approach most effective? What training methods are best? In what ways can online instruction support the acquisition of thinking skills? How can we best identify the thinking skills needed to achieve the business goals of our organization? These are the main questions we consider in this chapter.

Before discussing specific guidelines for building thinking skills, it makes sense to first ask whether there is any evidence that they can be enhanced through training at all and, if so, what types of training work best. Since the last edition of this book, a number of experiments have tested different approaches to teaching of thinking skills. For example, Marin and Halpern (2011) compared an infusion with an explicit approach to teaching generic thinking skills. An infusion approach involves integrating thinking skills into ongoing courses, whereas explicit training implies separate courses or lessons devoted specifically to thinking skills.

In the explicit training, a group of learners was randomly assigned to complete a web-based course of four sessions each focusing on a specific skill: analyzing arguments, understanding causal and correlational claims, forming mental models, and making sound decisions. Following each online tutorial, the instructor led a class discussion. The infusion group completed a course in cognition and cognitive development during which the instructor used critical thinking activities involving analysis of data, graph interpretation, observation of correlation, identification of cause and effect, and others. Outcomes were measured with a standardized thinking test with questions similar to those shown in Figure 16.2.

Chart shows sample thinking test item content and a question based on the content with YES or No option.

Figure 16.2 A Sample Thinking Test Item.

From Marin and Halpern, 2011.

Both groups improved their pretest scores compared to a control group that received no instruction. Of the two trained groups, the explicitly trained group outperformed the infusion group with a medium effect size of .45. The research team concludes: “Our studies reflect the benefits of an explicit mode of teaching critical thinking—making specific strategies abundantly clear to students. . . “ (p. 12).

Because a number of experiments similar to the Marin and Halpern study have been published, Abrami, Bernard, Borokhovski, Waddington, Wade, and Persson (2015) had sufficient data to conduct a meta-analysis on how to teach thinking skills. After reviewing and narrowing research studies, they calculated a moderate mean effect size of .30 from 341 effect sizes. As we discussed in Chapter 3, we can have more confidence in the results of a meta-analysis than any one experiment since the conclusions are based on a large number of experiments. Upon categorizing the various experiments, the research team found that critical skills training can be effective among all educational levels and for diverse types of subject matter. They also found domain-specific thinking skills training to be effective, with an effect size of .40.

What types of instruction led to best results? Based on an evaluation of the instructional techniques used in experiments with better learning outcomes, the research team recommends an emphasis on expert modeling of problem solving and student dialog (oral or written) in which students analyze real-world problems. Dialog can involve discussing or debating a problem in instructor-led whole-class or small groups. Exposure to authentic problems and examples was effective, particularly when learners were engaged in problem solving or role playing. Further in this chapter we describe multimedia problem-based training programs in more detail.

The results of this meta-analysis confirm a previous comparison of high and low effective thinking skills programs by Mayer (2008a). In this analysis, he notes that successful programs (1) focus on a few well-defined skills, (2) contextualize those skills within authentic tasks, and (3) incorporate social learning strategies, including instructor modeling and student collaboration. We conclude that thinking skills programs can be effective but, as with other skill training, job-specific thinking skills must be defined and trained with an emphasis on explicit instruction.

To help you design or select programs that are likely to give you a return on investment, we offer the following guidelines:

Principle 1: Focus on Explicit Teaching of Job-Relevant Thinking Skills

A number of research studies conclude that attention devoted to thinking skills within regular courses (the infusion approach) is not sufficient. Instead, the evidence leads us to recommend explicit training of thinking skills. Figure 16.3 includes an example of web-based instruction on argument analysis from the Marin and Halpern (2011) research reviewed in previous paragraphs.

Chart shows a content based on Analyzing arguments-deciding what to believe with six session topics such as conclusions, persuade reasons, recognizing reasons, reason categories, assumptions and analyzing.

Figure 16.3 Part of an Introduction to an Online Tutorial on Argument Analysis.

Adapted from Marin and Halpern, 2011, p. 8.

As you review the lesson introduction in Figure 16.3, note that the thinking skills (argument analysis) are contextualized in a real-world scenario to establish the relevance of the lesson. Second, review the lesson topics noting the focus on specific aspects of argument analysis, concluding with practice on the skills described.

Drawing on instructional methods reviewed in previous chapters, we recommend you design lessons that combine worked examples with engagement as follows: (1) display expert thinking models, (2) focus learner attention to the thinking behaviors of those models, and (3) promote active engagement with those models.

Display Expert Thinking Models

Successful thinking skills instruction teaches those skills in an explicit manner. Take a look at Figure 16.4 from our pharmaceutical consultative sales course. The sales expert is modeling the best responses to the physician’s statements and questions. In this example, the learner can see expert behaviors as well as gain insight into the expert’s thinking process. The on-screen bubble displays her thoughts as she frames her answers. Pressing the continue button leads to the remainder of the dialog. Expert thoughts could include consideration of alternative responses as in this example, a rationale for a response, and responses to avoid.

Screen shot shows names of two women as example and both of them arguing in an office room with a play button to resume.

Figure 16.4 The Thought Bubble Displays Expert Thinking Processes.

Focus Learner Attention to Behaviors of Expert Models

Moreno (2009) compared learning of teaching principles such as techniques to maintain attention, promote active learning, and prevent cognitive overload from lessons featuring animated teacher models that did or did not add focusing statements. Sixty one student teachers were assigned to a multimedia lesson that explained teaching principles, followed by an animated classroom model of an expert teacher applying the principles. In one lesson version, a narrative statement from the teacher summarizing the principle to be shown in the animation was placed just prior to the scene modeling that principle. For example, “To maintain students’ attention, I called them randomly by name throughout the lesson” would be heard just prior to seeing the animated model calling on various students. The comparison lesson version used the same animation but omitted the focusing narration. Moreno (2009) found that the group lacking the focus statements took significantly longer to study the animated models and scored substantially lower on a transfer test. She concludes that “virtual classroom exemplars should be carefully designed to include narrated guidance that can help prospective teachers make meaningful connections between the theory learned and the rich classroom information contained in the exemplars” (p. 499). In Figure 16.5 you can see how we applied this technique to our sales lesson.

Screen shot shows names of two women as example and both of them discussing their views in an office room with a play button to resume.

Figure 16.5 The Sales Representative Tells the Learner What to Watch for in the Video Example.

Promote Active Engagement with Expert Models

Van Gog, Sluijsmans, Joosten-ten Brinke, and Prins (2010) describe a pilot teacher-training online program in which learners select a professional situation such as handling groups of learners, conducting parental consultations, or asking effective questions. For each scenario reflecting a specific situation, learners are assigned to observe, analyze, describe, and act. In the observe task, the learner watches a video example of a teacher responding to the situation and writes summaries of the main actions taken. The analyze task uses the same video but requires the learners to evaluate the actions they identified during the observation. For the describe task, learners observe the start of a new scenario related to the same professional situation and describe how they would respond. The learner receives feedback by comparing an expert response for the observation, analysis, and description assignments. The final assignment requires the learner to respond to a similar situation on the job and receive feedback from a peer or mentor. Teachers who tested the pilot program gave it positive reviews. However, no learning data was reported.

Gartmeier and colleagues (2015) compared acquisition of communication skills among medical students and teachers from three conditions: (1) e-learning modules that followed a presentation of communication guidelines with learner analysis of effective and ineffective video communication examples, (2) face-to-face role play, or (3) a combination of e-learning and role play. They found both the e-learning and the role play groups demonstrated better communication skills on a performance test than a control group that received no training. The best learning occurred in groups that used both the e-learning and role play. However, if only one approach was used, the e-learning module was found to be more effective than the role-play lesson. Even though the role-play groups engaged in active practice, the research team suggested that an analysis of carefully selected effective and ineffective performance videos offered a more structured learning opportunity.

Considering the two previous studies, you might construct an online program that first explains and illustrates effective thinking or problem-solving guidelines, followed by analysis of effective and ineffective application examples (video examples in the case of communication skills). Follow the examples with practice that requires the learner to apply the guidelines or skills both in the training setting and in the workplace.

Principle 2: Design Lessons Around Authentic Work Tasks or Problems

The Abrami, Bernard, Borokhovski, Waddington, Wade, and Persson (2015) meta-analysis reports that best learning outcomes for thinking skills arise from instruction that involves exposure to authentic problems coupled with oral or written discussion of those problems. Their findings recommend an approach known as problem-based learning, case-based learning, or scenario-based learning, in which learning of thinking skills is based on job-relevant problem situations. In this section we will review three examples of problem-focused lessons followed by a list of their key features, contrasted with traditional directive instructional approaches.

Example 1: Problem-Based Learning (PBL)

About fifty years ago, McMaster’s University in Canada initiated PBL in their medical school curriculum, which subsequently has been widely adopted as an alternative to a traditional science-focused approach. In PBL, the science lectures that predominated the first two years of medical school are replaced by small team reviews of medical cases such as the example in Figure 16.6. Typically, a team of five to seven students facilitated by a faculty member reviews a case together and reaches a common understanding of the case followed by individualized self-study to learn more about the issues in the case. After a period of time, the team reconvenes to debrief lessons learned. Most PBL sessions follow a structured process such as:

  1. Clarify unknown terms and concepts.
  2. Define the problem in the case (critical thinking).
  3. Brainstorm to analyze the problem by identifying plausible explanations (creative thinking).
  4. Critique explanations produced and draft a coherent description of the problem (critical thinking).
  5. Define the learning issues (metacognitive thinking).
  6. Engage in self-directed study to fill the gaps specified by the learning issues (metacognitive thinking).
  7. Reconvene to debrief the case and share lessons learned (communication skills, metacognitive thinking).
Chart shows a short content on Miserable life of a stomach.

Figure 16.6 A Case Problem Used in PBL.

From Schmidt and Moust, 2000.

Many evaluation efforts have been directed at PBL, often comparing learning and motivation between PBL and the traditional curriculum. We review this research later in this chapter.

Example 2: Automotive Troubleshooting

In Figure 16.7 we show the interface for a multimedia scenario-based practice environment for automotive troubleshooting. The task assignment begins with a work order that states the symptoms of a malfunction, such as high idle. The learners can conduct tests using the virtual shop equipment to identify the source of the failure. Once they believe they have identified the fault, they can select their answers from a list of about fourteen different failures. When they have completed the case and resolved the failure, the learners compare their diagnostic decisions and repair actions with those of an expert, as shown in Figure 16.8.

Screen shot shows inside view of an automotive shop which includes a car, laptop, stool, table, digital clock, writing board and other equipments.

Figure 16.7 A Multimedia Interface for Automotive Troubleshooting.

With permission from Raytheon Professional Services.

Example 3: BioWorld

BioWorld is a multimedia environment designed to teach scientific reasoning processes, including evidence gathering and analysis. Originally designed for high school students, BioWorld was subsequently adapted for medical students (Lajoie, 2009). As shown in Figure 16.9, BioWorld displays a text description of a patient case. The learner begins by selecting relevant phrases mentioned in the case description and dragging them into the evidence table located in the left frame. For example, in this case involving a complaint of abdominal discomfort, the learner has selected patient age, complaint, and recent dietary changes. After identifying relevant evidence, learners select an initial hypothesis from the “Select Hypothesis” pull-down menu located in the upper left-hand corner. In this example, the learner selected salmonella. The learner can then order diagnostic tests from a pull-down menu to support the hypothesis. Learners can access resources from the online library at any time, including information on biological terms, diagnostic tests, and symptoms. At the conclusion of a case, the learner prioritizes the evidence supporting the diagnosis and can compare his or her priorities to those of an expert.

Screen shot shows Expert comparison window which displays customer concern, expert repair with diagnostic period of 1:15 hours and tool choices with diagnostic period of 2:40 hours.

Figure 16.8 A Comparison of Learner with Expert Problem-Solving Actions During Automotive Troubleshooting.

With permission from Raytheon Professional Services.

Screen shot shows Bioworld window where the evidence table page displays the concerns of a patient on abdominal discomfort to a doctor.

Figure 16.9 The Learner Moves Relevant Data into the Evidence Table in BioWorld.

With permission from S. Lajoie.

BioWorld includes many elements of an effective thinking skills program. First, it focuses on a medical reasoning model. Second, it is case-based. The learning is contextualized within the process of gathering evidence about a patient and forming diagnostic hypotheses. Third, it makes scientific reasoning explicit by requiring participants to select a hypothesis and build and prioritize evidence to support it. Fourth, it offers instructional support in the form of library resources. Fifth, BioWorld provides feedback on the accuracy of the hypotheses as well as the prioritization of evidence.

Features of Problem-Focused Instruction

A full description of problem-focused design elements can be found in Scenario-Based e-Learning (Clark, 2013). For the purpose of this chapter, we summarize four of the main features that distinguish this approach from a traditional directive training methodology.

  1. Problem-Centered. Learning starts with a job-realistic scenario or problem, as shown in Figures 16.7 and 16.9. Case studies are not new to training. However, in a traditional approach, the case study is sequenced at the end of a lesson or series of lessons. In contrast, in problem-focused instruction, the lesson is initiated by a case scenario that serves as the context for learning.
  2. Guided Learning. Learners are supported during the problem-solving episode to avoid mental overload. In directive designs, component lesson topics are sequenced one at a time in a building block fashion to avoid mental overload. To minimize overload in problem-focused learning, the design must manage the complexity of the scenarios as well as the amount of help available. Early lessons begin with a simple scenario, the solution for which might be demonstrated by an expert. Later lessons include complex scenarios with more variables and require the learner to do most of the work.
  3. Freedom to Make Mistakes. Problem-focused lessons may vary regarding the amount of freedom learners have to try different approaches and learn from them. More guided lessons, such as the branched scenario shown in Figure 16.10, offer fewer options and immediate feedback on choices. More open lessons, such as the automotive troubleshooting example and BioWorld, use a guided discovery approach in which the learner can try a number of actions and may not receive feedback until they submit a case resolution.
  4. Feedback may be explanatory, as described in Chapter 13. In addition, feedback may be intrinsic. By that we mean that, after taking an action, the learners may see the consequence of their action. For example, in the automotive troubleshooting lesson, an incorrect response results in the feedback you see in Figure 16.11. At the end of the case, a summary of student problem-solving actions is displayed next to the actions of an expert, as shown in Figure 16.8.
  5. Acceleration of Expertise. Using multimedia scenarios offers learners the opportunity to build experience faster than is often available in a traditional training or on-the-job environment. For example, troubleshooting a failure in the shop may require three or four hours of work, compared to twenty minutes in an online simulation.
Screen shot shows a nurse monitoring an anesthetic patient using anesthetic chart and patient information.

Figure 16.10 A Branched Scenario Design to Teach Anesthesiology.

With permission from Veterinary Information Network.

Screen shot shows a window displaying a clock.

Figure 16.11 Intrinsic Feedback Given to an Incorrect Response During Automotive Troubleshooting.

With permission from Raytheon Professional Services.

Evidence for Problem-Focused Instruction

As we mentioned in the introduction to this chapter, a comprehensive meta-analysis of critical thinking skills training recommends the use of authentic problems coupled with discussion (Abrami, Bernard, Borokhovskie, Waddington, Wade, & Persson, 2015). Not all problem-focused instruction, however, has yielded better learning outcomes compared to traditional directive approaches.

Evidence from Problem-Based Learning

Because problem-based learning is a widely adopted alternative in medical education, many studies have compared outcomes among medical students who studied in a PBL curriculum with medical students who studied in a traditional lecture science-based curriculum. Conclusions have varied. For example, Schmidt, Van der Molan, te Winkel, and Wijnen (2009) reported a meta-analysis of 270 research studies comparing outcomes between PBL and traditional medical students in a single medical school. They conclude that medical knowledge and diagnostic reasoning were generally equivalent between the two groups. In contrast, interpersonal skills, practical medical skills, and student satisfaction ratings favored the problem-based learning approach. Koh, Khoo, Wong, and Koh (2008) reviewed thirteen studies that assessed post-graduate medical competencies, comparing physicians who studied via PBL with those who studied under a traditional program. Assessment scores showed that the social dimension, including team work skills, appreciation of social and emotional aspects of health care, as well as communication skills, were higher among PBL graduates. There were no differences for other competencies.

In contrast, Albanese (2010) concludes that “Research on the effectiveness of PBL has been somewhat disappointing to those who expected PBL to be a radical improvement in medical education. Several reviews of PBL over the past twenty years have not shown the gains in performance that many had hoped for” (p. 42).

The success of PBL may depend on how it is implemented. For example, Kumta, Tsang, Hung, and Cheng (2003) found better learning among senior-year medical students who completed three online case scenarios each week than among those who participated in the normal curriculum. In this experiment, 163 medical students were randomly assigned to complete the traditional three-week program in orthopedics, consisting of formal lectures, bedside tutorials, and outpatient clinics, or to a test program that included the traditional program plus eight computer-based clinical case simulations. Those in the test program benefited from the additional experience the online scenarios provided, compared to those who completed the regular program of instruction. This research suggests that you can accelerate expertise by adding scenario-based instruction to an existing training program.

Although the effects of PBL on learning and medical competencies have been mixed, most reviews agree that, overall, students rate PBL more favorably than the traditional curriculum. Perhaps learning in the context of real-world patient cases makes the relevance of the lesson more salient and hence increases motivation. However, keep in mind that medical students are a unique population whose learning preferences may not match your audience.

Evidence from Sherlock

Sherlock is a computer-coached whole-task practice environment focused on troubleshooting realistic failures in the electronics of an F-14 aircraft. Sherlock was designed to provide automated apprenticeship-like training for airpersons who completed their technical school training. Similar to the automotive troubleshooting example we described previously in this chapter, the Sherlock environment emulated the real shop and provided a practice environment in the context of realistic troubleshooting assignments. An Air Force evaluation of Sherlock found that trainees who were on the job for six months and spent twenty to twenty-five hours working with Sherlock were as proficient in troubleshooting electronics problems as technicians who had been on the job four years (Lajoie, 2009).

This acceleration of expertise stems no doubt from the compressed experience that Sherlock offered. In the real-world troubleshooting environment, failures were infrequent and occurred in no specific order of complexity. In other words, the real world did not provide the optimal frequency and sequence of problems for learning. An important lesson learned from Sherlock is the opportunity to accelerate expertise through experience with digital cases that in the real world could take months or years to accumulate.

A Summary of Evidence for Problem-Focused Instruction

Problem-focused instruction has been shown to benefit the acquisition of thinking skills in a number of experiments (Abrami, Bernard, Borokhovskie, Waddington, Wade, & Persson, 2015). However, the specific features of an effective design may depend on the learners and domain goals. We look forward to additional research to: (a) define the situations under which problem-focused designs are more effective than traditional directive designs, (b) identify important design elements that make problem-focused designed effective for learners of varied prior knowledge, and (c) identify how directive and problem-focused designs can best be combined.

Principle 3: Define Job-Specific Thinking Processes

As you plan e-learning that uses a problem-focused approach, build in case scenarios, research tools, data sources, activities, and thinking processes that reflect job-specific expert approaches to problem resolution. Review the generic thinking skills summarized in Table 16.2 as a start. Can any of these skills be relevant to specific job roles in your organization? If so, can you adapt them to reflect the thinking skills specific to your domain?

You can identify these job-specific thinking skills during the analysis phases of the design process. Because most experts cannot articulate their rationale when asked, you will often need to use special techniques called cognitive task analysis (CTA) to define the scenarios to be solved in the training, as well as the thinking skills experts use to solve them.

In Table 16.3 we summarize several cognitive task analysis techniques. Which technique will work best for you will depend on the nature of the problems being solved as well as the work environment. For example, concurrent reporting, which requires the workers to talk aloud while they resolve a task, cannot be used for a task that requires talking, such as sales or customer service, or for tasks that cannot be observed, such as a combat situation. For tasks such as these, a retrospective approach that asks experts to later recall their actions and thoughts may be more appropriate. For more details on cognitive task analysis, refer to Scenario-Based e-Learning by Clark (2013) and Working Minds by Crandall, Klein, and Hoffman (2006).

Table 16.3. Some Cognitive Task Analysis Methods.

Method Description Tradeoffs
Concurrent Reporting Subjects asked to verbalize all of their thoughts at the same time that they are solving a problem or working on a task. Not practical with verbal tasks such as sales Obtrusive May provide high amount of relevant data
Retrospective Reporting Subjects asked to verbalize all of their thoughts immediately or soon after solving a problem or working on a task. Relies on memory Unobtrusive
Cued Retrospective Reporting Subjects asked to verbalize all of their thoughts after solving a problem or working on a task while viewing a record (video recording, eye-tracking data) of their work. Provides memory support Obtrusive
Critical Decision Method Expert identifies and reports on a past incident in which he or she solved a problem or worked on a task. Probing questions asked throughout several interview iterations. Relies on memory Unobtrusive
Structured Expert Interview Several experts independently describe three situations of diverse complexity in which they resolved a given professional situation and list the factors that influence their complexity rating. A consensus meeting identifies complexity factors and most appropriate response to situations. Relies on memory Leverages multiple sources of expertise Unobtrusive

What We Don’t Know About Teaching Thinking Skills

Based on evidence to date, we have recommended some specific instructional approaches for helping learners build job-relevant thinking skills. However, many questions remain:

  1. For what kinds of learners and work tasks will a job-focused lesson be more effective than a traditional directive lesson?
  2. How can traditional directive approaches to teaching thinking skills be integrated with problem-focused exercises?
  3. How can problem-focused learning environments accommodate the evolving expertise of a learner?
  4. How will design of problem-focused cases or lessons differ for relatively well-structured problems such as automotive troubleshooting compared to more open problems that have multiple approaches and solutions?
  5. What types of guidance are most effective to avoid extraneous mental load in problem-focused lessons?
  6. What is the potential return on investment (ROI) for the time invested in cognitive task analysis and design of thinking-skills e-learning?
  7. How can off-the-shelf thinking skill courseware be adapted to support job-specific contexts?
  8. What are some effective methods to elicit productive learner dialog during problem-focused learning?

Chapter Reflection

  1. How important are thinking skills to (a) your organization, (b) specific job roles within your organization, and (c) education of citizens in a high-technological society?
  2. For a job role or academic domain familiar to you, what kinds of instructional methods might you consider to build critical thinking skills?
  3. How effectively can e-learning support thinking skills? What other media might you consider as part of a blended solution?

COMING NEXT

Since our previous edition, the number of valid research studies on games and simulations has increased, yielding the beginnings of some evidence-based design guidelines. In the next chapter we review what has been learned about the effects of games on cognitive skills and game design features shown to improve learning.

Suggested Readings

  1. Abrami, P.C., Bernard, R.M., Borokhovski, E., Waddington, D.I., Wade, A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85, 275–314. This report is a lengthy and technical review of evidence on critical thinking skills. We recommend it for anyone interested in an in-depth discussion and analysis of the research.
  2. Clark, R.C. (2013). Scenario-based e-learning. San Francisco: Pfeiffer. This book focuses exclusively on the design and development of various forms of problem-focused multimedia lessons for workforce learning.
  3. Crandall, B., Klein, G., & Hoffman, R.R. (2006). Working minds. Boston, MA: MIT Press. A practical book that provides guidelines and examples of conducting cognitive task analysis in the workplace.
  4. Gartmeir, M., Bauer, J., Fisher, M.R., Hoppe-Seyler, T., Karsten, G., Kiessling, C.L., Moller, G.E., Wiesbeck, A., & Prenzel, M. (2015). Fostering professional communication skills of future physicians and teachers: Effects of e-learning with video cases and role play. Instructional Science, pp. 1–20. A useful research report for anyone interested in training communication skills.
  5. Lajoie, S.P. (2014). Multimedia learning of cognitive processes. In R.E. Mayer (Ed.), The Cambridge handbook of multimedia learning. (2nd ed.; pp. 673–646). New York: Cambridge University Press. This chapter focuses on ways that multimedia technology has been used to build thinking skills.
  6. Marin, L.M., & Halpern, D.F. (2011). Pedagogy for developing critical thinking in adolescents: Explicit instruction produces greatest gains. Thinking Skills and Creativity, 6, 1–13. This research report is a representative experiment comparing an explicit with an infusion approach to teaching critical thinking skills.
  7. Mayer, R.E. (2008). Learning and instruction (2nd ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall. See Chapter 12: Teaching by fostering problem-solving strategies.

CHAPTER OUTLINE