11.6. When to ask a diagnostic question

What I used to think

When I did use diagnostic questions, it would only be at the beginning of the lesson, mainly as a way of assessing some of the skills students would need for the rest of the lesson.

It turns out that this was like having one of the world’s most powerful computers and only using it to play solitaire.

Sources of inspiration

My takeaway

There are four different times or scenarios that I now use diagnostic questions, with each serving its own purpose.

1. Start of the lesson

The start of the lesson is the most obvious time to use a diagnostic question, but for four different reasons.

a) To assess baseline knowledge

In a 2016 article on feedback, Wiliam wrote a line that really resonated with me: ‘We need to start from where the learner is, not where we would like the learner to be’.

One of the key features of the models of thinking that we discussed way back in Chapter 1 was that existing knowledge makes acquiring new knowledge a whole lot easier. This makes attempting to discover where students are – and in particular which misconceptions they hold – before you attempt to teach them new knowledge one of the most important parts of teaching. Hodgen and Wiliam (2006) advise that teachers should ‘start from where the learner is, recognising that students have to reconstruct their ideas and that to merely add to those ideas an overlay of new ideas tends to lead to an understanding of mathematics as disconnected and inconsistent’.

But deciding which baseline knowledge to assess can be easier said than done.

A few years ago, I inherited a wonderful Year 11 class who suffered from a severe and debilitating lack of confidence. When I taught them vectors, I put in extra effort as it was one of the few topics they did not have any negative preconceptions about because they had not encountered it (and failed at it) before. However, all my hard work was undone when I gave my students an exam-style question like this:

Figure 11.21 – Source: GCSE Maths Takeaway, available at http://mrbartonmaths.com/students/legacy-gcse/gcse-maths-takeaway.html

My students could not do Part b); not because they did not understand vectors, but because they were not secure in dealing with the addition of whole numbers and negative fractions that is required to get the final answer. Had I instead thought carefully about all the skills involved in mastering vectors – not just the obvious ones – I could have prevented this. I could have asked them a question like the following at the start of the lesson:

Figure 11.22 – Source: Craig Barton for Diagnostic Questions

This would have revealed their weakness and I could have addressed it completely separately from our study of vectors, providing explanations, examples and practice as required. Instead, their confusion with this relatively small skill became tangled up in their wider perception of vectors, and hence vectors became yet another topic in maths that they could not do. Likewise, me subsequently trying to teach them this skill alongside navigating through a complex vectors problem likely imposed a significant burden on their working memories, leading to cognitive overload, with little learning of either fractions or vectors taking place. And this was 100% preventable.

I have found it beneficial when planning a lesson to make a list of all the prerequisite knowledge that students need to have acquired and automated in order to have the best chance of acquiring the new knowledge you want to teach them. It is very similar to the process suggested when we discussed Deliberate Practice in Chapter 8. Once again, this is a particularly useful thing to do as part of a departmental meeting when discussing an upcoming topic – everyone makes their own list, and then you compare. Our curse of knowledge makes this a difficult exercise, and so the more help the better. For example, without help from my department I would not have thought to include a question on placing decimals on a number line before teaching histograms (this skill is, of course, required in order to accurately plot frequency density).

b) To prime students’ long-term memories

The assessing of baseline knowledge serves another purpose that is of direct benefit to the student – their long-term memory is primed for the acquisition of the related new knowledge you intend to teach them. Mccrea (2017) explains that ‘we can increase the chances of learning happening by activating, or warming-up relevant areas of long-term memory before exploring a new topic’. This is a major benefit of asking good questions. They force the student to direct attention to the relevant topic, and essentially start the process of retrieving and organising relevant knowledge in their long-term memory that will make the acquisition of new knowledge so much easier. So, if I want to teach my Year 10s how to add algebraic fractions, some diagnostic questions at the start of the lesson on adding non-algebraic fractions, simplifying expressions and expanding double brackets will not only allow me to assess their understanding of these relevant skills and intervene accordingly, but also bring these skills to the forefront of students’ minds so they are ready to be applied. I think of priming like an athlete doing stretches in preparation for a race – the correct areas are warmed up and the body (or, in our students’ case, the mind) is more prepared to accept the challenge of what is to come.

c) To make use of the Testing Effect

The Testing Effect is related to priming, and will be covered in much more detail in the next chapter. But in essence, when students are compelled to retrieve knowledge, that very process modifies and enhances what is stored in long-term memory (eg see Karpicke and Blunt, 2011). This is, of course, good news if that knowledge is needed for the forthcoming lesson, but it can also be useful to ask questions on unrelated concepts that cover something students did last week, month or year. Given how quickly these diagnostic questions can be asked and answered, you can get through five or more in a matter of minutes. Setting up a schedule to ensure that all prior skills and concepts are revisited throughout the year via these questions is one of the best habits I have developed. The only downside is what to do if students get the question wrong. If it is not crucial to the content of the reset of the lesson, I tend to be honest with my students, reassuring them not to worry, that it is best we found that out now, and that we will come back to it some time soon. As discussed in the last section, I have seen and delivered too many lessons that got derailed by an unrelated starter activity that did not go as planned. If I do not need to deal with it now, then I don’t, and instead come back when I am more prepared.

d) To make use of the Pretest Effect

When I interviewed Robert and Elizabeth Bjork for my podcast, one of the most fascinating concepts they discussed was the Pretest Effect. This is where being tested on material you have not studied before actually enables you to better understand and remember the material when you do study it. Whereas priming refers to focusing students’ attention on material they have seen before that is related to what they are about to learn, pretesting is for content that is brand new. We will discuss pretesting in more detail next chapter, but two studies merit a quick mention here. Richland et al (2009) found that generating a wrong answer increases our chances of learning the right answer, and Kornell et al (2009) found that as long as students are given appropriate feedback, you will still see a testing benefit even if they get the answer wrong on a pretest. Hence, as counterintuitive as it sounds, asking a diagnostic question at the start of the lesson on something students have not encountered before but will be doing this lesson, and having the subsequent discussion to ensure students are clear what the correct answer is, may well have two benefits. It will give you an insight into who knows what, which will better inform your teaching. But secondly, it may actually aid your students’ ability to understand the new topic. More on this – and the potential pitfalls – next chapter!

2. Middle of the lesson

Imagine you are halfway through a particular lesson. The students are working on some carefully prepared sequence of questions (Chapter 7) or Purposeful Practice activity (Chapter 10). They are all at different points, working at their own pace. You are trying to make an informed decision whether or not now is the right time to move onto the next part of the lesson, maybe to introduce the next skill or concept. One of the best ways to do this is to ask a well-designed diagnostic question.

William (2011) describes diagnostic questions used in this ways as ‘hinge questions’ (Fun fact: see what happens if you visit hingequestions.com). He explains:

The central idea here is that the teacher designs each lesson with at least one ‘hinge’ in the instructional sequence. The hinge is a point at which the teacher checks whether the class is ready to move on through the use of a diagnostic question. How the lesson proceeds depends on the level of understanding shown by the students, so the direction of the lesson hinges at this point.

Diagnostic questions presented in the way suggested earlier in this chapter are perfect for this as they are quick to ask, quick to get data back from, and do not simply tell you where certain students are going wrong, but why they are going wrong. This allows you to intervene where needed. Conversely, if everyone gets the question right, then onwards and upwards!

3. End of the lesson

As discussed in Section 3.10, I no longer end my lessons on a difficult question. For me, a key purpose of the end of the lesson is to inform the starting point of the next lesson. I want to get a quick and accurate sense of my students’ understanding of what I have taught them that particular lesson so I know my starting point the next time I see them. Hence, I use a diagnostic question.

Of course, we need to bear in mind that the distinction between learning and performance warns us against inferring that students have definitely learned everything when we assess them immediately after instruction. However, wrong answers certainly tell us there are problems we need to address, and the specific nature of those problems. Likewise, if everyone gets the last question correct (and it is a good one!), then I have evidence to move on next lesson, but crucially making a note to revisit the concept at numerous points in the future.

A common practice amongst teachers is to issue an Exit Ticket at the end of the lesson. This is essentially a question that students complete in the final minutes of the lesson and hand to the teacher on the way out. The teacher then has a quick look over the students’ responses and uses this to inform their planning. Diagnostic questions are perfect for this purpose. They are lightning fast, both for the students to complete and for the teacher to mark. I have started printing them out and adding a section for the student to explain their answer. This enables me to glance at their choice of A, B, C and D, and then dig deeper into the reason behind this choice if I need to. It also allows the students to reap the benefits from self-explaining discussed in Chapter 5. An exit ticket might look like this:

Figure 11.23 – Source: Craig Barton

Finally, the end of the lesson is also an opportunity to give students a question based on what they will be studying next lesson. This serves a dual purpose – it allows me to get a sense of their understanding of key concepts so I can adjust my planning accordingly, and it also provides an opportunity to take advantage of the Pretest Effect outlined above.

4. Homework

As we have seen, writing good diagnostic questions is flipping hard…which makes it the ideal activity to give to your students! Students must demonstrate a sound understanding of the key concept or skill, together with an appreciation of where other students may go wrong.

A homework following a lesson on sharing in a ratio may look like this:

The same homework can be set to all students, giving each the opportunity to demonstrate the depth of their knowledge of a concept. Such a homework also has the advantages of being relatively quick for me to mark, illuminating in terms of the insights I can learn, and may provide me with a new batch of diagnostic questions that I can use with my students. Everyone’s a winner.

What I do now

Ciofalo and Wylie (2006) write: ‘There is flexibility both in terms of when in a lesson a diagnostic item might be used and how a teacher might use it. The most important thing to consider is that the teacher is evoking information she needs to inform subsequent instruction or to provide feedback to students’.

The information I receive from my students’ responses to diagnostic questions at various points in a lesson is vital. It allows me to respond in real-time to their ever-changing needs. The information is quick to collect and accurate. As such, the use of diagnostic questions has probably become the most important part of my teaching.

If only everyone agreed with me…