Behaviors observed during administration of subtests can be used to enhance hypothesis testing about what cognitive constructs are, or are not, being used to perform subtests. This appendix discusses behavior observations related to each subtest within each domain.
Special considerations apply when attempting to determine the extent to which subtest performance is being affected by a lack of use of executive functions. The examiner needs to keep in mind that direct observations of behavior often cannot determine the extent of involvement of executive functions in task performance. It is necessary to generate and test hypotheses regarding the role of executive functions in poor subtest performance. This is because poor performance on any subtest may be caused by a number of executive function difficulties. Within each domain, a list is provided of executive functions that could adversely affect performance if not utilized when needed.
When an examinee performs poorly on a subtest, follow-up with testing of the limits can help to determine whether the poor performance is caused by a lack of facility with the cognitive constructs that are being assessed or the poor performance is caused by a lack of use of executive functions needed to effectively perform the task. The process involved for testing the limits is to readminister the subtest while providing additional executive function scaffolds intended to cue the examinee as to the cognitive constructs needed to perform the task. At the least intrusive level, the examiner asks the examinee a vague, general question about task performance intended to offer a very subtle cue about what the examinee should do to perform the task as effectively as possible. In the case of the Similarities Subtest, this first question could take the form of a reflective question. For example, when returning to the Similarities subtest after completing the standard administration of the WISC-V, the examiner could state the following, “You answered several questions correctly, but when you got to ____ and ____, you said, ‘I don't know ’ okay, but let me ask you a question: ‘Is there any way that you could know the answer?’” This vague cue is intended to help the examinee realize that there might be other ways to think about the item that might lead to a correct solution. If the examinee grasps the intention of the cue, she or he is likely to engage the reasoning capacities needed to at least make an effort to figure out how the two words are alike. If the examinee does not pick up on the first cue, additional, more specific cues can be provided in a scaffold-like manner in an effort to prompt the examinee to engage in reasoning with language.
Regardless of the score obtained, when an examinee performs inconsistently (difficult items completed correctly, but relatively easy items completed incorrectly), it is highly likely that executive functions are not being applied as effectively as possible. In these instances, additional information should be gathered to confirm or refute hypotheses about executive function difficulties.
When taking the Verbal Comprehension subtests, examinees should record all responses as close to verbatim as possible and note the amount of time elapsed before he or she offers a response and the amount of time required to deliver a response. Beyond the recording of verbal responses, examiners should note additional aspects of the examinee's behavior, including:
This recorded information can help examiners develop and test hypotheses about the mental constructs that are, or are not, being used to produce an examinee's responses.
It is critical to keep in mind that the same specific behavior can have different implications regarding the cognitive constructs being engaged. For example, a long latency period followed by a relatively inarticulate response to an item from the Similarities subtest often signals the accessing of reasoning ability to generate a response, but such hesitancy and inarticulateness may indicate a need for more time to try to organize a coherent verbal response because of expressive language difficulties or difficulties with the effective accessing of information from long-term storage.
It is also important to recognize that many examinees will not be uniform in their use of a specific cognitive capacity as they progress through the items of a single subtest. Cognitive capacity use will vary depending on the examinee's perceptions and judgments about what constructs are needed to produce an accurate response; the cognitive constructs most likely to be required to complete the item effectively (these can vary greatly for the items of a single subtest); the degree to which various cognitive constructs are accessible to the examinee; and the degree of competency the examinee is able to demonstrate in the use of various cognitive constructs. For example, in the case of the Similarities subtest, a majority of examinees will be reflexively retrieving stored associations when responding to the easiest items (these easy items naturally elicit such reflexive associative responding). As items get more difficult and likely to be less familiar, the examinee needs to use executive functions to become aware of the need to engage reasoning abilities and cue the shift from retrieval to reasoning in an attempt to respond effectively.
In the case of the Vocabulary subtest, items will be more uniformly perceived as requiring the retrieval of previously learned information from long-term storage. Occasionally, however, an examinee faced with an unfamiliar word might attempt to reason out the word's meaning; such attempts usually, but not always, fail even in cases where the examinee has a wealth of knowledge about prefixes, suffixes, word roots, and the derivations of words from other cultures.
An examinee who struggles to provide an adequate response to the most basic of word relationships on the Similarities subtest may be demonstrating the effects of (1) a poor store of knowledge; (2) a poor ability to reason with verbal concepts; (3) poor knowledge and poor reasoning ability; (4) adequate stored knowledge but difficulties with retrieval of that knowledge; or (5) adequate reasoning and retrieval constructs but compromised expressive language abilities.
In contrast, an examinee may have stored so much information about word relationships that he or she goes much further into the Similarities subtest using only retrieval from long-term storage. When challenged by much more difficult items to use reasoning to “figure out” a relationship on the spot, the examinee now avoids engaging reasoning by simply saying “I don't know.”
Hypotheses to be considered in such a situation include:
Determining the most likely match between behavior and cognitive capacity use requires diligent testing of hypotheses; the more carefully an examinee's behavior is observed and the more a consistent pattern of behavior emerges within one subtest as well as across several subtests, the more likely the generation of a confirmable hypothesis. In many cases, testing the limits after completion of all WISC-V subtest standardized testing can help to confirm or refute specific hypotheses.
Behaviors related to the specific cognitive constructs most likely to be used with the tasks of the Verbal Comprehension Index are described below.
The capacity to reason with verbal information is most likely to be assessed by an item when an examinee does not immediately offer a very specific response and/or requires more than 1–3 seconds to offer the response to the items of a subtest that requires the use of reasoning. Examinees who need to reason out the relationship between the two words (Similarities subtest) or think about the reasons for social conventions of behavior (Comprehension subtest) often offer longer, less articulate, more convoluted responses during which they add more information as they speak that either clarifies the relationship or introduces a new line of reasoning that may or may not be more accurate than the initial portion of the response. The use of the prompt “Tell me more” will be required with examinees who need to apply reasoning, because their explanations may be vague and nonspecific. Examinees who resist the effort needed to engage reasoning processes or who are ineffective at cueing themselves to apply their reasoning abilities will be more likely to offer quick “I don't know” responses. When observing a person who is attempting to use reasoning, one feels a qualitative sense of mental effort that comes through in the behaviors displayed.
The capacity to retrieve information from long-term storage is most likely to be assessed when the examinee shows no hesitation in initiating a response and provides an immediate, concise response such as offering a synonym for the meaning of a word (Vocabulary subtest), a basic fact (Information subtest), stating a single concept word or brief phrase that conceptually links the two stimulus words of an item (Similarities subtest), or reflexively reciting a very concise, well-organized, well-articulated rationale or offering a seemingly rote phrase to explain the reason(s) for social conventions of behavior (Comprehension subtest). When observing a person who is retrieving information from long-term storage effectively, one usually feels a qualitative sense of effortlessness that comes through in the behaviors displayed (e.g., relaxed body posture and facial expression).
For some examinees with generalized or specific retrieval problems, however, there may be signs of great mental effort as they attempt in vain to recall the meaning of a word (Vocabulary subtest), a relationship between words that they believe they know (Similarities subtest), or an isolated fact (Information subtest). Some examinees will verbally express great consternation with such struggles to retrieve information that they are certain they have stored at some time in the past. These difficulties with recall can have a more generalized effect of shaking the examinee's confidence in her or his ability to engage the cognitive constructs needed to perform the various WISC-V Subtest tasks, including those that do not require the retrieval of information from long-term storage. When observing a person who is having difficulty retrieving information from long-term storage, one usually feels a qualitative sense of struggle that comes through in the behaviors displayed (e.g., hunched body posture, looking upward for prolonged periods of time, and contorted facial expressions such as frowning, brow furrowing, or grimacing).
Examinees who have difficulty with one or more of the auditory processes required for accurately perceiving and representing auditory input are at risk of offering incorrect responses because of subtle misperceptions or even a complete lack of comprehension of what was said when directions and/or individual items were being presented.
Examinees who are experiencing difficulties with auditory acuity, auditory attention, auditory discrimination, or auditory comprehension, or who have slow auditory processing speed, often engage in behaviors that reflect these difficulties, such as:
The greater the number of times these behaviors occur, the greater the likelihood that auditory process deficits are present. Hypothetically, the effects of auditory process deficits can be minimized through repetition of directions and items; this hypothesis can be confirmed as accurate when repetitions of directions and items are followed by correct responses, indicating that initial difficulties with auditory processing were the most likely source of the initial confusion about the auditory input or the cause of the initially incorrect response. Hypotheses regarding the specific nature of auditory process deficits cannot be fully tested using only the information obtained in administration of the WISC-V, necessitating the use of additional measures specifically designed to assess the use of auditory processes. Special note should be made of attention difficulties that might indicate the need for ADHD (Attention-Deficit/Hyperactivity Disorder) or ADD (Attention Deficit Disorder) assessment.
For all of the Verbal Comprehension subtests, the examinee is required to provide a vocal response. In the case of the Vocabulary, Information, and Similarities subtests, full credit can be earned for many items simply by stating a single word. For the remainder of the items of these subtests and all of the items of the Comprehension subtest, responses can vary from a single phrase to multiple sentences. For some examinees, low item scores may reflect an inability to adequately articulate information they were able to retrieve from long-term storage or an inability to articulate the product of their reasoning. It is important to note, however, that responses need not be well phrased or grammatically, syntactically, or morphologically correct in order for an examinee to earn full credit. As a result, some examinees with significant expressive language difficulties can score in the average range or above on any of the four Verbal Comprehension subtests. In such cases, picking up on and justifying the need for a more detailed speech and language evaluation will depend more on the careful observation and description of the examinees expressive language difficulties than on the attainment of one or more low subtest scores.
Specific difficulties observed during administration of the Verbal Comprehension subtests that might reflect an inability to express the product of reasoning or information that was retrieved, or that might indicate the need to refer for further assessment of speech and language on the basis of poor quality or poor production of language include the following:
The short directions and one- or two-word item stimuli of the Similarities and Vocabulary subtests make few working memory demands on examinees. Some of the Information subtest items and most of the Comprehension subtest items are much longer utterances and therefore somewhat more susceptible to the effects of weak working memory capacity. It is possible that examinees may find it difficult to hold the question in mind while attempting to locate the desired information in long-term storage (Information) or while attempting to construct an adequately reasoned response (Comprehension). When such disruption occurs, the most common result is a distorted response that does not really fit the original question. Based on the content of the response, the examiner may be able to identify what part of the question was not retained during mental efforts at retrieval or reasoning. If working memory disruption is suspected, it will be necessary to test such a hypothesis by finding ways to rule out the possible effects of other cognitive constructs, including difficulties with auditory acuity, auditory discrimination, auditory comprehension, and/or auditory attention.
The highly specific subtest directions of the Verbal Comprehension subtests do much to reduce unwanted executive function involvement in the process of understanding what is required in order to perform items effectively. Even the highly structured format of the WISC-V, however, cannot completely eradicate the need for engagement of executive functions during task performance. Observance of the following behaviors may be indicative of difficulties with some of the executive functions involved in Verbal Comprehension Index task performance that are listed in Rapid Reference 6.6:
When administering the Fluid Reasoning and Visual Spatial subtests, examiners should record all responses and note the amount of time elapsed before the examinee offers a response and other response behaviors such as changing responses.
This cognitive construct is essential for effective performance of most of the items of the Matrix Reasoning, Block Design, and Figure Weights subtests and some of the items of the Picture Completion subtest. It is important to keep in mind that reasoning with nonverbal visual stimuli does not necessarily involve nonverbal reasoning. Reasoning with visual stimuli can involve either nonverbal or verbal reasoning, or some combination of the two. Examinees often can be observed talking to themselves about Matrix Reasoning and Figure Weights items, and occasionally talking themselves through the placement of blocks on the Block Design subtest or trying to talk themselves through the spatial perception needed to complete Visual Puzzles items. The type of reasoning abilities that are engaged to perform Fluid Reasoning or Visual Spatial tasks will depend on the individual examinee's preferences and/or his or her perceptions of what abilities are required to do the task.
Timing items enables the examiner to test hypotheses about the kinds of reasoning that are being employed by the examinee. Quick accurate performance of items of the Matrix Reasoning and Figure Weights subtests (e.g., under 5 seconds for relatively easy items, under 10 seconds for mid-range difficulty items, under 15 seconds for difficult items) strongly suggests that the examinee is using nonverbal reasoning to complete these items because the engagement of verbal reasoning increases the amount of time required to complete items. As completion times increase, however, the more difficult it becomes to determine whether the additional time is caused by slow perception and organization of the visual information presented or by the use of language to assign verbal labels to the visual elements of the item. Overt verbalizations by the examinee will help to confirm the latter hypothesis, but in the absence of such overt cues, testing beyond the WISC-V would be needed to further test these hypotheses.
Although the visual materials used with the WISC-V have been designed for ease of viewing, examinees who have difficulty with one or more of the visual processes required for accurately perceiving and representing visual input are at risk of offering incorrect responses because of subtle misperceptions or even a complete lack of understanding of what they are looking at when attempting to perform individual subtest items.
Examinees who are experiencing difficulties with visual acuity, visual attention, visual perception/representation, or visual discrimination often engage in behaviors that reflect these difficulties, such as:
The greater the number of times these behaviors occur, the greater the likelihood that visual process deficits are present. The effects of some visual perception and discrimination problems (or the effects of poor executive direction of visual perceptions and discrimination) can be tested using a testing of the limits approach (after completion of standard administration of the WISC-V) wherein the examiner “perceptually guides” the examinee through an organized input of the visual information on the easel page. An example of this technique would be guiding the examinee through Matrix Reasoning by facilitating the examinee's visual processing of each cell in the item and each item response. For each Matrix Reasoning item, point to each image as you say the following: “Look at the first one here. This one goes with this one the way this one goes with which one of these? Look at the first choice. Now look at the second choice; now look at this one, now look at this one; now look at this one. Now, which one of these [point to each of the response choices] goes here?” Hypotheses regarding the specific nature of visual process deficits cannot be fully tested using only the information obtained in administration of the WISC-V; it is necessary to use additional measures specifically designed to assess the use of visual processes. Special note should be made of attention difficulties that might indicate the need for Attention-Deficit/Hyperactivity Disorder or Attention Deficit Disorder assessment.
Being able to generate visual images “in the mind's eye” and hold and manipulate such visual images may be essential to the performance of the more difficult items of the Matrix Reasoning, Visual Puzzles, and Figure Weights subtests and certainly can enhance greatly an examinee's performance with many or all items of the Block Design subtest. It is certainly conceivable that an examinee with very poor visualization and/or very poor working memory capacities can succeed with most or all of the Block Design and Picture Concepts items, many of the Matrix Reasoning items, and at least some of the Visual Puzzles and Figure Weights items by frequently and repetitively rescanning the visual stimuli of the items. In the case of Block Design, an examinee with poor visualization and/or poor working memory can frequently compare the design model with the construction in progress to judge the accuracy of block placements and guide further performance. Observation for testing hypotheses about the use of working memory involves careful monitoring of the examinee's eye movements and gaze fixations and timing of individual item performance. Examinees who are attempting to by-pass the use of, or compensate for a lack of, working memory resources will exhibit frequent shifting of gaze between various locations on the easel page, especially between elements of the item stimulus and the multiple choice response options. The greater the number of changes in gaze fixation, the more likely it is that the examinee is having difficulty holding enough visual information in working memory to enable efficient problem solving, thereby necessitating continual re-registering of visual elements for use in working memory. These inefficiencies will increase the amount of time required to work on an item, thereby reducing the likelihood of success with difficult items that have strict time limits (Visual Puzzles, Figure Weights, and the more difficult Block Design items).
In contrast, examinees who are making the most efficient use of working memory tend to scan the stimuli to initially register and represent the item's visual contents, then shift gaze to the response options at the bottom of the page (or to the blocks), rarely, if ever, looking back up at the item or block model before delivering a correct response. When such behavior is observed, it strongly supports the hypothesis that the examinee was able to hold the initially registered stimuli or block design in working memory while choosing from among the response options or while constructing an accurate copy of the Block Design model.
Careful observation can help identify when it is necessary to test hypotheses about the extent to which performance with Fluid Reasoning and Visual Spatial tasks is being adversely affected by working memory difficulties. The Figure Weights subtest is especially well suited for testing hypotheses through adaptive readministration procedures wherein the examinee is allowed to use pencil and paper to aid in problem solving. Improvement of performance using pencil and paper increases the likelihood that the relatively poorer performance during the standard administration of the WISC-V was caused by the need to hold information in working memory rather than an inability to apply quantitative reasoning with visually presented nonverbal stimuli. Similar adaptive procedures could be used with Matrix Reasoning and Visual Puzzles items when behavior observations suggest that these subtests also are being adversely affected by a lack of use of working memory.
Completion of Block Design items requires adequate use of fine motor movements for accurate placement of the blocks into the properly oriented 2 × 2 or 3 × 3 configuration. Examinees who may be lacking in motor coordination for a variety of reasons may find it difficult to handle the blocks, resulting in inaccurate block placements, rotated or fragmented designs, slowed production and disruption of reasoning and other capacities because of an over-focusing on motor coordination. Careful observation is required to determine the full impact of such difficulties on performance of each item.
Although the WISC-V Technical and Interpretive Manual (Wechsler, 2014) states that the effect of processing speed on Fluid Reasoning and Visual Spatial Index scores has been reduced due to a reduction of the number of Block Design items offering bonus points for speed (four items instead of six), the use of per item time limits on all Fluid Reasoning and Visual Spatial Index subtests except Matrix Reasoning and Picture Concepts makes processing speed a critical element in performance in this domain. This manual also points out that time limits were necessary to obtain item difficulty gradients with the Visual Puzzles and Figure Weights subtests, noting that many more examinees were able to perform many of the items of these subtests when no time limit was enforced. This fact strongly supports the hypothesis that processing speed, independent of reasoning ability or visual perception and discrimination, can greatly affect task performance. Contrasting scores on the Matrix Reasoning subtest and Picture Concepts with scores on the other FRI and VSI subtests can offer some evidence for the effect of processing speed (see Rapid Reference 6.9). Item-level observations and timing of individual item performance, however, can offer even stronger support when slow processing speed is thought to be affecting performance.
When an examinee is requiring relatively longer periods of time to complete Matrix Reasoning items (e.g., 10 seconds or more for the easiest items, 20 seconds or more for mid-range items, and more than 30 seconds for difficult items) and “times out” on multiple Figure Weights, Visual Puzzles, and/or Block Design items, hypotheses regarding the effects of slower processing speed can be tested using a testing of the limits approach after standard administration of the WISC-V by readministering items without time limits. When such procedures are used, completion times should be recorded; knowing how much time is required for obtaining correct responses offers a great deal of insight into the impact that strict, short time limits may have on the examinee.
Full consideration of the issue of the effect of processing speed on FRI and VSI subtest performance requires a discussion of the “30-second guideline” for administration of Matrix Reasoning subtest items. The WISC-V Administration and Scoring Manual indicates examiners can use clinical judgment in determining how long to allow an examinee to work on any individual Matrix Reasoning item, but 30 seconds per item can be used as a guideline in order to maintain a reasonable pace of administration. Examiners therefore must make judgments as to how long examinees are allowed to work on items. Adhering too closely to the 30-second guideline may restrict some examinees' opportunities for successful completion of items. Carefully timing and recording individual item performance times can provide important information that can be used in the process of deciding how much time to allow an examinee on each item. When a slow but accurate work pace is established early, examiners should consider allowing more than 30 seconds for difficult items when the examinee appears to be actively engaged with problem-solving efforts.
The highly specific subtest directions of the Fluid Reasoning and Visual Spatial subtests do much to reduce unwanted executive function involvement in the process of understanding what is required in order to perform items effectively. The directions of the Matrix Reasoning subtest were revised to offer greater structure to examinees and the number of different Matrix Reasoning item types was reduced from four to two, thereby reducing further the need for examinees to “figure out” for themselves what kind of problem-solving strategies should be engaged. Even the highly structured format of the WISC-V directions and items, however, cannot completely eradicate the need for engagement of executive functions during task performance. Observance of the following behaviors may be indicative of difficulties with some of the executive functions involved in Fluid Reasoning task performance that are listed in Rapid Reference 6.7:
Examiners also should consider readministration of multiple-choice tasks when examinees demonstrate an inconsistent pattern of responses (e.g., easy items answered incorrectly but difficult items answered correctly) without any behaviors indicative of impulsive responding.
When administering the Working Memory Domain subtests, and/or the Arithmetic, Symbol Translation subtests, examiners should record all responses verbatim and note the amount of time elapsed before the examinee offers a response as well as the amount of time required to deliver a response. Beyond the recording of verbal responses, examiners should note additional aspects of the examinees' behavior, including the quality of the responses provided in terms of the use of intonation patterns, prosody and phrasing; interpersonal social communication style, body position, and comments about the testing process or statements about their perceptions of what is being required of them; and emotional reactions. This recorded information can help examiners develop and test hypotheses about the mental capacities that are, or are not, being used to produce an examinee's responses.
Determining the most likely match between behavior and cognitive construct use requires diligent testing of hypotheses; the more carefully an examinee's behavior is observed and the more a consistent pattern of behavior emerges within one subtest as well as across several subtests, the more likely the generation of confirmable hypotheses.
Behaviors related to the specific cognitive constructs most likely to be used with the tasks of the Working Memory Domain and the Arithmetic and Immediate Symbol Translation subtests follow. Initial registration of auditory (or visual) stimuli and application of working memory with auditory (or visual) stimuli are the primary cognitive constructs intended to be assessed with all of these subtests. It should be noted, however, that an examinee's capacity for initially registering the stimuli presented and manipulating the information in working memory are reflected directly in the number of correct responses delivered only when no other cognitive constructs are constraining performance. This means that efforts first should be made to determine the adequacy of the use of secondary cognitive constructs that can act as constraints on task performance.
Examinees who have difficulty with one or more of the auditory processes required for accurately perceiving and representing auditory input are at risk of offering incorrect responses because of misperceptions or even a complete lack of registration of what was said when directions and/or individual items are being presented. Examinees who are experiencing difficulties with auditory acuity, auditory attention, auditory discrimination, or auditory comprehension, or who have slow auditory processing speed, often engage in behaviors that reflect these difficulties, such as:
The greater the number of times these behaviors occur, the greater the likelihood that auditory process deficits are present. Hypothetically, the negative effects of auditory process deficits can be minimized through repetition of items, at least in the case of some of the items of the Arithmetic subtest. For the Letter-Number Sequencing subtest, repetition of directions may help to improve task performance for examinees with poor auditory comprehension. Administration of the Digit Span and Letter-Number Sequencing subtest items, however, prohibit examiners from repeating items, thereby eliminating repetition as a means of testing hypotheses about the effects of auditory process deficits on item performance. For Digit Span and Letter-Number Sequencing items, the careful observation for the presence of the behaviors mentioned previously is the only way to generate and test hypotheses about the effects of auditory process deficits on task performance. Hypotheses regarding the specific nature of auditory process deficits however cannot be fully tested using only the information obtained in administration of the WISC-V, necessitating the use of additional measures specifically designed to assess the use of auditory processes when difficulties are suspected. Special note should be made of attention difficulties that might indicate the need for ADHD or ADD assessment.
Although expressive language response demands appear to be minimal for all of the working memory tasks, some examinees may experience difficulty with the demand for verbatim, sequential repetition or precise sequential reordering of the stimuli. Sequencing difficulties are readily apparent in the responses provided by examinees; individuals with sequencing difficulties often provide all of the correct numbers or letters and numbers that were presented but do not deliver them in the correct order. In other cases, sequencing errors are accompanied by omissions and/or additions. It is important for examiners to recognize the qualitative difference between incorrectly sequenced responses that offer all of the right numbers or numbers and letters and incorrect responses that fall short of providing all of the original stimuli. An examinee whose output reflects only a sequencing error was able to initially register and manipulate the information in working memory, but somewhere in the transition from initial registration to verbal output a misrepresentation of the information occurred. Examinees who are unable to reproduce all of the original stimuli are much less effective in their efforts at initial registration and/or in their use of working memory than those who make sequencing errors. Sequencing errors represent inefficient use of existing capacities rather than a lack of existence of such capacities.
Before engaging in the retrieval from long-term storage of knowledge about how to perform calculations and solve problems while holding the math problem in working memory, an examinee must have learned how to perform the necessary math calculations and have had some exposure to math problem solving exercises. If an examinee's math skills are deficient, no amount of working memory capacity will enable a correct response. When a low score is obtained on the Arithmetic subtest, multiple hypotheses can be generated: (1) poor math skills; (2) poor initial registration capacity; (3) poor working memory capacity; (4) poor retrieval of math knowledge from long-term storage; (5) a combination of two or more of any of these four. Although the WISC-V does not offer a means for effectively testing these competing hypotheses, the WISC-V Integrated will provides this capability. Without the WISC-V Integrated, examiners can use improvised testing of the limits which would involve re-administration of the Arithmetic items in various formats (showing printed versions of the items while the examinee attempts to complete the item; offering pencil and paper to use to figure out answers; providing written math calculation problems that represent the math calculations involved in each item).
The highly specific subtest directions of the Working Memory subtests do much to reduce unwanted executive function involvement in the process of understanding what is required in order to perform items effectively. Even the highly structured format of the WISC-V, however, cannot completely eradicate the need for engagement of executive functions during task performance.
Qualitatively poor effort or an overly nonchalant or overly confident attitude toward the Working Memory tasks may indicate difficulties with cueing the appropriate considerations of the cognitive demands of the Working Memory Subtest tasks and the amount of mental effort required to effectively perform the tasks. Problems with gauging the amount of effort needed and/or initiating effort and/or sustaining effort to produce accurate responses can result in inconsistent performance within a task (getting longer span items correct after failing shorter span items; incorrectly responding on the first or second trial of a span length while correctly responding on the other trial of that span length) or across tasks (performing better with Digit Span Backward and/or Digit Span Sequencing than with Digit Span Forward; performing better with Letter-Number Sequencing than with Digit Span).
When performance on Digit Span Backward and/or Digit Span Sequencing is better than performance on Digit Span Forward, the most likely source of difficulty is the poor use of one or more of the executive function capacities needed to cue and direct efficient performance. Such paradoxical results most often involve poor use of one or more of the following executive function capacities:
Although any of these six executive function capacities could be involved in paradoxical performance with Digit Span items, the likelihood that problems will occur only with Digit Span Forward items and not Digit Span Backward items is unlikely for the last four capacities listed. Some combination of the first two executive function capacities listed, however, is highly likely to be the source of difficulty when performance on Digit Span Backward is significantly better than performance on Digit Span Forward. Many examinees underestimate the difficulty involved in merely repeating digits forward but accurately perceive the difficult nature of repeating digits in reverse order; as a result, they do not cue the appropriate consideration of the cognitive demands of Digit Span Forward and/or do not cue the appropriate level of mental effort needed to effectively perform Digit Span Forward. These examinees under10activate resources for Digit Span Forward then appropriately activate resources for Digit Span Backward with the result being better performance on Digit Span Backward than Digit Span Forward. Similar performance paradoxes may be observed with any of the Process Subtest span tasks when they are used in an assessment.
The adaptive use of executive functions to improve performance can be seen when examinees offer responses in chunked units as reflected in their intonation and phrasing patterns. For example, an examinee who is provided the digits 8-1-4-9-2-6 delivered in a monotone with about 1 second between each responds with prosodic groupings of “814, 926” has just demonstrated the use of executive functions to cue the use of a chunking strategy to reduce the difficulty of the item. In essence, the examinee was able to reduce a six registration-slot item into a two registration-slot item by employing this strategy, thereby increasing the likelihood of performing better overall than if the strategy had not been cued.
All of the tasks of the Working Memory Index require the examinee to initially register auditory stimuli. When an examinee appears to be free of auditory acuity, discrimination and/or comprehension difficulties, appears to be attending to the presentation of each item, and a consistent response pattern is obtained, the examiner can hypothesize with greater certainty that the number of items answered correctly represents the examinee's capacity for initially registering the type of stimuli presented (i.e., random number series, random number/letter series, math word problems).
All of the tasks of the Working Memory Index except Digit Span Forward require the examinee to hold and manipulate auditory stimuli. When an examinee appears to be free of auditory acuity, discrimination and/or comprehension difficulties, appears to be attending to the presentation of each item, and a consistent response pattern is obtained, the examiner can hypothesize with greater certainty that the number of items answered correctly represents the examinee's capacity for holding and manipulating in working memory the type of stimuli presented (i.e., random number series, random number/letter series, math word problems).
When administering the Processing Speed subtests, examinees should closely observe the examinee's efforts at task completion. For the Coding and Symbol Search subtests, close observation is enhanced greatly through the use of time interval response recording procedures.
Time interval response recording can be used with the Coding and Symbol Search subtests to assess the extent to which an examinee is able to focus and sustain effort and/or direct the focusing and sustaining of effort for simple, repetitive tasks for a period of 2 minutes. Time interval response recording involves counting and recording the number of items completed during specific time intervals throughout the total 2-minute total time period. Although examiners can choose the time intervals they use (e.g., 15-second or 30-second), normative data have been collected with other editions of the Wechsler Scales (e.g., WISC-IV Integrated, 2004) using 30-second time intervals.
To obtain time interval data, the examiner records the number of symbols coded for the Coding subtest or the number of items marked for the Symbol Search subtest at the end of each 30-second interval after subtest timing begins. The 30-second response recording interval approach enables the clinician to view a set of four data points that can be used to evaluate the examinee's response pattern.
The clinically relevant information that can be gleaned from the recording of time interval production is the pattern of performance across the full 2-minute work period. Unfortunately, the 30-second response recording procedure was not used to collect data during the standardization of the WISC-V, precluding the offering of 30-second time interval raw score means and standard deviations for either the Coding or Symbol Search subtest, but these data likely will be included in the WISC-V Integrated.
Although the visual materials used with the WISC-V have been well designed for ease of viewing, examinees who have difficulty with one or more of the visual processes required for accurately perceiving and representing visual input are at risk of offering incorrect responses because of subtle misperceptions or even a complete lack of understanding of what they are looking at when attempting to perform individual subtest items. Examinees who are experiencing difficulties with visual acuity, visual attention, visual perception/representation, or visual discrimination often engage in behaviors that reflect these difficulties, such as:
The greater the number of times these behaviors occur, the greater the likelihood that visual process deficits are present. The effects of some visual perception and discrimination problems (or the effects of poor executive direction of visual perceptions and discrimination) can be tested using a testing of the limits approach (after completion of standard administration of the WISC-V) wherein the examiner “perceptually guides” the examinee through an organized input of the visual information on the easel page.
Hypotheses regarding the specific nature of visual process deficits cannot be fully tested using only the information obtained in administration of the WISC-V; it is necessary to use additional measures specifically designed to assess the use of visual processes. Special note should be made of visual attention difficulties that might indicate the need for Attention-Deficit/Hyperactivity Disorder or Attention Deficit Disorder assessment.
Although all three of the Processing Speed subtests require examinees to use a pencil to provide responses, the specific grapho-motor demands of each Subtest vary greatly. The Coding Subtest places the greatest demands on grapho-motor capacities, requiring continuous production for the entire 2-minute period. Slowed performance, poorly formed symbols, or coding errors may result from motor coordination difficulties and or motor fatigue. Examiners should also look for signs of difficulties with basic pencil-handling skills, such as poor pencil grip or extremely heavy or light pressure on the pencil. Use of the time interval response coding procedures described previously can be useful in helping to identify examinees who are exhibiting grapho-motor difficulties despite earning scores in the average or above average range. Such examinees may start out performing very effectively for one to three 30-second intervals but then drop to and remain at a very low level of production for the remaining intervals. When such sudden drops in performance are noted, competing hypotheses regarding grapho-motor fatigue or difficulties with executive function cueing and directing of sustained effort should be considered.
Being able to generate visual images “in the mind's eye” and hold and manipulate such visual images can enhance an examinee's performance with all of the Processing Speed subtests. For the Coding subtest, movement can be reduced and time might be saved if the examinee can hold in mind the digit-symbol associations shown in the coding key. The revised format of the WISC-V Coding subtest however, wherein each of the digits 1–9 appears randomly with equal frequency in each row of top boxes, makes it less likely that examinees will adapt the use of working memory capacities for the WISC-V Coding as quickly as they may have with the WISC-IV, where digits 1–4 are repeated more frequently at the beginning of the task and digits 5–9 are gradually introduced into the mix later. Examiners who observe task performance closely will be able to note the point at which the examinee no longer looks up at the code key or back at his or her own work, but rather relies completely on working memory for access to the digit-symbol associations. In contrast, examinees who make little or no effort to engage visual memory beyond basic initial registration of the digits in the top box will continually look back up at the code to determine the digit-symbol relationship to fill in each empty box.
Effective use of visual memory capacities can aid examinees in the quick inspection of Symbol Search items. Careful observation of examinee eye and head movements during performance will reveal the frequency with which the examinee refers back to the target symbols while inspecting the response option symbols. Examinees who are extremely effective at applying visual memory capacities will show a slight latency period while initially registering the target symbols, then move to inspect the response symbols and make their choice without ever looking back at the target symbols. In contrast, examinees having difficulty initially registering and holding the target symbols in mind while inspecting the response options will frequently look back and forth from target to option.
Careful observation of examinees' behaviors associated with the use of, or lack of use of, initial registration and working memory capacities applied to visual stimuli during performance of the PSI subtest tasks can provide valuable information that can be compared with information from other subtest observations for the purpose of testing hypotheses about memory capacity use or disuse.
Inarguably, examinees who earn above-average scores on a Processing Speed subtest have demonstrated above average processing speed. Such scores, however, do not preclude the possibility that the examinee is inefficiently and/or inadequately employing his or her processing speed capacities. Additionally, examinees who earn scores in the average and below ranges also may be inefficiently or inadequately employing their processing speed capacities, resulting in underestimates of these capacities. Use of the time interval recording procedures described previously can be very useful in helping to generate hypotheses about how an examinee is, or is not, employing his or her processing speed capacities and the cognitive difficulties that might be interfering with efficient production. Because each of the PSI subtests requires the use of processing speed capacities across a somewhat prolonged time frame, the effects of cognitive capacities other than processing speed often become more pronounced as time progresses. These effects can be quantified and explored using the time interval recording procedure.
Although the highly specific subtest directions of the Processing Speed subtests do much to reduce unwanted executive function involvement in the process of understanding what is required in order to perform items effectively, executive functions will need to be engaged in order for an examinee to sustain attention and effort consistently for the duration of a Processing Speed task and to avoid making errors. Use of the time interval response recording procedures can help greatly with the generation and testing of hypotheses about the effects of executive function difficulties on task performance.
Qualitatively poor effort or an overly nonchalant attitude toward the timed aspect of the Processing Speed tasks may indicate difficulties with cueing the appropriate considerations of the cognitive demands of the Processing Speed subtest tasks and the amount of mental effort required to perform the tasks effectively. Examinees with such difficulties are likely to produce consistently low scores across all four 30-second response intervals. Problems with gauging the amount of effort needed and/or initiating effort and/or sustaining effort to produce accurate responses can result in inconsistent performance within a task. When an examinee's efforts do not produce a pattern of consistent numbers of responses within time intervals, one of the most likely sources of difficulty beyond grapho-motor control problems is the poor use of one or more of the executive function capacities needed to cue and direct efficient performance. Following are the executive function capacities most likely to be involved in task performance and the observed behavior patterns obtained from the interval response recording that are most closely related with each:
These difficulties often result in consistently low production across intervals despite observed use of faster processing speed for other WISC-V subtests or other tasks administered in the assessment or reports from other sources. When these difficulties are short term, problems with initially activating to the required level may be observed. Slow gauging of task demands and/or slow initiation of effort will result in decreased production in the first to third intervals relative to performance in the remaining 30-second intervals after the relatively slower start.
These difficulties often result in inconsistent production across the four time intervals of the task.
Some examinees are adept at actively engaging executive function capacities in an effort to make task performance more efficient and effective. These examinees will alter their work strategies from that suggested in the subtest directions, such as employing working memory resources to hold the Coding digit-symbol associations in mind while working instead of continuously looking back at the code key for each item, or referring back to own work instead of looking up at the coding key. For Symbol Search, some examinees choose to divide the task into two separate search routines, scanning the response options once to look for the first target and then again to look for the second target if a match was not found for the first target. During tryouts of the Cancellation subtest, many examinees attempted to employ a “corn-rowing” strategy of working across the first row from left to right and then working back across the second row in a right to left direction to ensure that they would maintain their place and not lose time in the process of reorienting by moving all the way back to the left-hand side of the page to start the second row. Specific directions had to be inserted into the standardization directions to ensure that this strategy was not employed by examinees—an alteration that reduced the likelihood that use of these executive functions would be affecting Cancellation performance. In contrast, some examinees do only what they are told and do not make any effort to further engage executive functions to alter their response strategy.
Some examinees experience great difficulty with the cueing and directing of the use of motor routines to complete fine motor tasks involving the use of a pencil. The observed effect may be very poorly formed symbols or excessively long or short slashes or an inability to keep symbol drawings inside a single box, random search movements that do not contribute to task performance, or overly careful drawing of symbols.
Examinees may demonstrate difficulty with adjusting their performance to the demands of these speeded tasks, working too fast or too slowly despite indications that an appropriate work pace could be engaged as demonstrated with other tasks or reports. When an exceptionally fast work pace is countered by several task errors, the likelihood is great that the examinee was not adjusting work pace to meet the demands of the task. Conversely, examinees who engage in overly careful productions of symbols or slash marks and frequent checking of work for accuracy throughout the task even though they have been performing very accurately are likely to be having difficulty adjusting their work pace to match the relatively simple demands of the task.
Although the WISC-V manuals do not report the number of errors made by WISC-V standardization examinees, Coding subtest data from the WISC-III-PI reflected a surprisingly stable error rate for children ages 6 to 16. For the WISC-III Coding subtest, fewer than 2 percent of children at each age range made more than four to five errors, and fewer than 25 percent made more than one error. Assuming that examinees included in the WISC-V standardization sample are at least as capable as those in the WISC-III standardization sample, WISC-V examiners should expect to see similar error rates on the Coding subtest.
Similarly, using four errors as a rule of thumb for the Symbol Search subtest would provide a very conservative estimate for an excessive error rate considering the fact that the typical production rate for Symbol Search items was about half that of coding items for the WISC-III standardization sample. Examinees that make an excessive number of errors on the Coding and/or Symbol Search subtests are likely having difficulty with cueing and directing the use of cognitive capacities involved in monitoring for errors and/or cueing the cognitive capacities involved in correcting errors once they have been detected.
Some examinees find it difficult to use executive functions to coordinate the multi-tasking demands of the Coding subtest. These individuals often show good processing speed and efficient performance with the Symbol Search subtest, but earn scores on the Coding subtest that are 3 or more scaled score points lower than the score they earn on the Symbol Search subtest. Examiners wishing to further test the hypothesis of multi-tasking coordination difficulties have the option of administering the WISC-V Integrated Coding Copy subtest. When the Coding Copy subtest score is 3 or more scaled score points better than the Coding subtest Scaled Score, there is a greater likelihood that difficulties with executive function direction of multi-tasking are negatively affecting performance on the Coding subtest.
Undetected errors on the Coding subtest accompanied by a relatively quick work pace are likely to be reflective of difficulties with inhibiting impulsive responding. Errors on the Symbol Search subtest that reflect selection of visual near-matches to target symbols accompanied by a relatively fast work pace also often can be indicators of a lack of inhibition of impulsive responding. When lack of inhibition is observed with other WISC-V tasks, however, the likelihood is that commission errors on the Cancellation subtest also are reflecting such difficulties increases proportionately with the number of Cancellation subtest commission errors being made.
Examiners may find it useful to note when an examinee who demonstrates the effective use of working memory with multiple WISC-V subtests does not appear to be applying working memory resources to hold the digit-symbol relationships in mind while working on the Coding subtest.
The naming speed subtests require quick and efficient retrieval of verbal labels. Because most children age 6 and older have been using the verbal labels associated with the sizes, colors, objects, letters and numbers, and quantities used for these items, retrieval is tapping into highly automated naming processes that are part of an examinee's remote long-term storage. To perform these tasks effectively, the examinee must have adequate expressive language, visual processing, attention, and processing speed capacities. Difficulties with any of these will result in less than optimal performance.
For each Naming Speed task, examiners are instructed to record the completion time, which is then used as the raw score for deriving a normative scaled score. Examiners need to be aware of the fact that although only completion time is used to derive a scaled score, there can be substantial qualitative differences among examinees in terms of the accuracy of performance, and these variations in accuracy of responses are not factored into the obtained scores. It is essential, therefore, that the examiner carefully record the number of errors made by the examinee and compare this number to the base rates obtained with the standardization sample (Table C.18 in the Administration and Scoring Supplement). Additionally, examiners should note the occurrence of the following behaviors, as these offer great insight into difficulties that a score based solely on completion time may be masking. Also recommended is the practice of recording the completion time for each row of stimuli within an item so as to have a better understanding of the pace of production throughout task performance.
It is important to note that the Naming Speed Literacy tasks, unlike the Naming Speed Quantity tasks, are not simply measures of the speed of automaticity of retrieval from singular modules within remote long-term storage (i.e., rapid automatic naming tasks of a single dimension, such as naming letters only). Rather, these tasks require the examinee to shift between modular stores of automated information. Effective performance of these tasks therefore requires the use of executive functions to direct the shifting between the knowledge stores as quickly as possible. Inefficient use of executive functions can result in several distinct types of errors with these tasks as described following.
When poor performance is observed on all items of the Naming Speed Literacy task that are administered (two tasks for ages 6–8; only one task for ages 9–16), hypotheses related to rapid naming difficulties should be tested by administering more basic Rapid Automatic Naming (RAN) tasks that require the naming of a single attribute (e.g., letter names only, numbers only, object names only, color names only). If poor performance is observed on these more basic RAN tasks, attention should be focused on basic language deficits rather than on executive control of rapid automatic switching.
Relatively slow, but error-free performance with Naming Speed Literacy across the Size-Color-Object items and Letter-Number items for ages 6–8 and with Letter-Number naming for ages 9–16 is likely to be reflecting a relative lack of automaticity with the retrieval of verbal labels.
Successful performance of the Naming Speed subtests depends in part on the physical capacity to conduct oral-motor movements in a coordinated manner. Difficulties with oral-motor sequencing are likely to result in one or more of the following:
Most examinees within the WISC-V age range are able to handle the expressive language demands of the Naming Speed tasks. When asked to label size, color, and object name, most examinees will naturally follow the example of the examiner and find it most effective to perform the naming activity in the order of size, then color, then object name. This natural flow of size-color-object represents an innate sense of grammar and syntax that appears to be shared by native speakers of the English language and is quickly imitated by examinees who speak English as a second language.
Examiners may occasionally encounter an examinee who is unable to maintain, or who does not use at all, this natural grammatical sequencing of attributes (e.g., saying “red, big, door” or “car, little, yellow” or “big, duck, blue”). When this type of expressive language difficulty is observed, the examinee should be referred for a more in-depth speech/language assessment.
Listed below are the executive function capacities most likely to be compromised in task performance and related behavior observations.
In some cases, the slow rate of production may be the result of the lack of use of executive functions (the Pace Cue) to increase the rate of production to match the instructions provided (“Name them as fast as you can without making any mistakes”). When an unusual number of errors are observed along with slow production time, additional difficulties involving the use of other executive functions are likely to be present. When naming errors are observed in concert with average or fast naming speed, the errors most likely are related to ineffective use of executive functions other than the Pace Cue.
The Speeded Naming Literacy subtest requires examinees age 7 and older to complete the Letter-Number Naming item that requires the use of executive functions to shift between naming numbers and naming letters. This is especially likely to be the case when performance on the Letter-Number switching task contains many more errors than performance on the Size-Color-Object task or other basic rapid automatic naming tasks. Difficulties with the direction of this rapid automatic shifting (typically referred to as rapid automatic switching) can result in any or all of the following:
Some examinees may find the performance of Speeded Naming tasks to be so exceptionally challenging that they are unable to maintain control of their actions during performance of these tasks. When an examinee cannot modulate his activity level effectively during task performance, one or more of the following behaviors are likely to be observed:
The occurrence of corrected or uncorrected errors is likely to reflect difficulties with the effective use of executive functions to monitor responses and to correct them when necessary. The number of errors made can be compared to the errors made in the standardization sample to determine whether or not the errors represent an unusual response pattern. The greater the number of errors, the greater the likelihood that executive function difficulties are present. Failure to monitor performance and/or to correct errors when they are detected are difficulties that can have a great impact on classroom performance. When excessive errors are noted, hypotheses related to executive function difficulties should be confirmed or refuted with additional testing or the gathering of additional information from situations outside of the testing environment.