Table 10.1 Research domain criteria matrix
Reproduced from NIMH Research Domain Criteria (RDoC), Draft 3.1: June 2011 <http://www.nimh.nih.gov/research-priorities/rdoc/nimh-research-domain-criteria-rdoc.shtml> (accessed May 7, 2014).
Scientific validity in psychiatry: Necessarily a moving target?
James Phillips
With the publication of DSM-5, it is clear that an important goal—diagnostic validity—was not achieved. The quest for this Holy Grail began back in 1980 with DSM-III (American Psychiatric Association 1980). The immediate goal of that manual was to achieve the first step in a scientific nosology—diagnostic reliability—with the use of operational definitions and diagnostic criteria. With DSM-III we could be confident that clinicians and researchers in different countries would be talking about the same phenomenon when they discussed, for instance, schizophrenia. Built into the DSM-III process, however, was the understanding that reliable diagnoses could not yet claim to be valid; we could not feel comfortable that the diagnostic concept in question represented a distinct, real entity in the world. How did we know, for instance, whether the diagnostic entity called schizophrenia described one distinct illness or several? In that way the accomplishment of DSM-III immediately unleashed a new anxiety and a new goal—securing diagnostic validity.
In prioritizing reliability over validity, the architects of DSM-III assumed that ongoing research would lead to valid diagnostic constructs. The standard of validity was the set of criteria proposed by Robins and Guze in their 1970 article (Robins and Guze 1970), along with subsequent publication of the Feighner diagnostic criteria in 1972 (Feighner et al. 1972), and Robert Spitzer’s Research Diagnostic Criteria in 1978 (Spitzer et al. 1978). The profound shock of the ensuing two decades was the inability, despite extensive research, to achieve the Robins/Guze criteria, exemplified by the failure to delineate the pathophysiology of the major psychiatric disorders, and the corresponding failure to find biomarkers for these disorders. The DSM-III-R, DSM-IV, and DSM-IV-R diagnostic categories remained plagued by fuzzy boundaries, excessive comorbidity, and excessive clinical use of NOS diagnoses. It turned out that psychiatric disorders are vastly more complicated than imagined by Robins, Guze, and other experts of the DSM-III era (Detre 1987).
In the face of this crisis of validity, the architects of DSM-5 recognized that a major goal of DSM-5 would be to accomplish what had not been accomplished with DSM-IV or DSM-IV-TR. As the DSM-5 leaders wrote in the 2002 white paper, A Research Agenda for DSM-V (Kupfer et al. 2002):
In the more than 30 years since the introduction of the Feighner criteria by Robins and Guze, which eventually led to DSM-III, the goal of validating these syndromes and discovering common etiologies has remained elusive. Despite many proposed candidates, not one laboratory marker has been found to be specific in identifying any of the DSM-defined syndromes. Epidemiologic and clinical studies have shown extremely high rates of comorbidities among the disorders, undermining the hypothesis that the syndromes represent distinct etiologies. Furthermore, epidemiologic studies have shown a high degree of short-term diagnostic instability for many disorders. With regard to treatment, lack of treatment specificity is the rule rather than the exception. (p. xviii)
The authors’ response to this crisis was an appeal for a “paradigm shift.”
All these limitations in the current diagnostic paradigm suggest that research exclusively focused on refining the DSM-defined syndromes may never be successful in uncovering their underlying etiologies. For that to happen, an as yet unknown paradigm shift may need to occur. Therefore, another important goal of this volume is to transcend the limitations of the current DSM paradigm and to encourage a research agenda that goes beyond our current ways of thinking to attempt to integrate information from a wide variety of sources and technologies. (p. xix)
As DSM-5 got under way, the leadership realized that ongoing research in the DSM-III/DSM-IV manner would not resolve the validity problem or produce the promised paradigm shift. With this realization in mind the leadership at first turned toward a large-scale, dimensional reframing of the diagnostic categories (Regier et al. 2011), and then later looked forward to the NIMH Research Domain Criteria project (RDoC) (Kupfer and Regier 2011) for possible answers to the validity crisis. Neither had an effect on the final DSM-5 document.
The final note in this background review is the DSM-5 field trials. They strike an ironic note, given that the challenge of DSM-5 was presumably to establish diagnostic validity, diagnostic reliability having been mostly accomplished in DSM-III and its successors (Kupfer 2002). What we find, however, is that the Field Trials are back to reliability. Oddly, validity was more or less taken for granted while the trials were being developed to again test reliability. In the reports of the trials, Clarke et al. write that “[t]he face and construct validity of the revised DSM-5 diagnoses were subjectively confirmed by the Work Groups that proposed the diagnostic changes” (2013: 43), and Regier et al. write that “[s]ince the 1970s, the validity of psychiatric diagnoses has largely been supported by expert clinical consensus, based on a wide range of clinical experience and increasingly buttressed by basic, clinical, and epidemiologic research” (2013: 59). (Regier et al. do acknowledge the limitations of established diagnostic validity, writing that “reliability studies set the stage for validity studies beyond face/construct validity” (2013: 59) but not clarifying how that will be achieved.)
The reasoning of the Field Trials appears to be: we cannot achieve full validity without firmly established reliability; we do not have the latter, so we have to start over with serious reliability studies; in the meantime, we have enough validity of sorts to warrant testing the existing and new diagnostic constructs for reliability.
If this reversal of priorities—assume validity, test reliability—is the first surprise from the Field Trials, the second is the results. Using the statistical methodology of kappa values for testing inter-rater reliability, the Field Trials tested 15 adult and 8 child/adolescent diagnoses. Five diagnoses were in the “very good” range, nine in the “good” range, three in the “unacceptable” range, and eight with insufficient sample sizes to generate kappa values.
Interpreting these results is a matter of perspective. The architects of the trials consider them a triumph. Indeed, it is arguable that they are on par with the reliability of general medical diagnoses (Pies 2007). On the other hand, it is striking that major depression and generalized anxiety disorder do not have acceptable kappa values. Given the enormous prevalence of anxiety and depression disorders, these findings seem like a big hole in the Field Trials.
Furthermore, while we should certainly acknowledge the limited success of the trials, we need also remember that they say little about diagnostic validity. Schizophrenia, schizoaffective disorder, and bipolar disorder, for instance, all do well in DSM-5 reliability—i.e., can be distinguished with diagnostic criteria—but we still don’t know whether they represent one disorder or three.
Leaving such questions regarding diagnostic validity aside, DSM-5 Chairman David Kupfer, along with Task Force member Helena Kraemer, wrote about the trials, “We ultimately tested the criteria for 23 disorders. The question we asked was a straightforward one: In the hands of regular clinicians, assessing typically symptomatic patients in no different way than they would during everyday practice, was a particular disorder reliable?” (Kupfer and Kraemer 2012: 10). They say nothing about the establishing of validity, and their conclusion leaves us questioning what we have accomplished with DSM-5—a handful of new diagnostic categories and hair-splitting changes of existing criteria sets, but, as with DSM-III and IV, categories with no established validity, and likely persistent problems of heterogeneous presentations, excessive comorbidity, and fuzzy boundaries. Thus several years of work by devoted Work Groups, ending with limited success in reliability, which we had thought were answered with DSM-III in 1980, and no word on validity.
In pursuing the question of validity of psychiatric diagnoses, we need to step back and examine how validity is defined. Validity can refer to the validity of the diagnostic criteria, validity of the external measures used to confirm diagnoses, and finally, validity of the diagnostic constructs themselves. These dimensions of validity are intertwined and can lead to circular reasoning. For instance, the validity of a diagnostic construct may be supported by its high scores on one or more external validators, but that then depends on the validity strength of the validators themselves, which in turn can be defended on the basis of their strong association with what seems like a desirable construct (Kendler 1990).
In most psychiatric discussion of diagnostic validity, validity tends to mean validity of the diagnostic constructs. This was Robins and Guze’s (1970) understanding of validity and implicitly that of DSM-III and all subsequent revisions. They judged validity with the use of their five validators—clinical description, laboratory findings, separation from other disorders, follow-up studies, and family history—but their emphasis was on validity of the diagnostic constructs, not the validity of their five validators.
In what follows I contrast two representative forms of diagnostic validity within the Robins/Guze framework, which I designate as strong and weak validity, the first, represented by Kendell and Jablensky (2003), focusing on validity of the constructs, the second, represented by the Guidelines (Kendler 2009), used by the DSM-5 Work Groups in their consideration of existing and proposed new diagnoses, focusing on validity of the validators.
Kendell and Jablensky distinguish two criteria for assessing validity:
We suggest, therefore, that a diagnostic category should be described as valid only if one of two conditions has been met. If the defining characteristic of the category is a syndrome, this syndrome must be demonstrated to be an entity, separated from neighboring syndromes and normality by a zone of rarity. Alternatively, if the category’s defining characteristics are more fundamental—that is, if the category is defined by a physiological, anatomical, histological, chromosomal, or molecular abnormality—clear, qualitative differences must exist between these defining characteristics and those of other conditions with a similar syndrome. (2003: 8)
For diagnoses that are defined only in terms of signs and symptoms (i.e., virtually all DSM diagnoses), Kendell and Jablensky highlight the third Robins/Guze criterion, “separation from other disorders,” which they redescribe as a “zone of rarity,” and declare that syndromal diagnoses are valid only if they meet that standard of strict separation from other disorders. For diagnoses that are defined at a more fundamental level, e.g., Huntington’s Disease, defined in terms of a dominant gene, the defining physiologic or neuroscientific features must be distinct from other disorders with similar presentations. Effectively, Kendell and Jablensky use two of the five Robins/Guze standards for asserting diagnostic validity: separation for other disorders for syndromal validity, and laboratory studies for more fundamental validity.
According to these authors, a diagnostic construct either meets these standards or it does not. They emphasize that all DSM syndromal diagnoses merge and blur to such a degree with other diagnoses that they do not meet the standard of syndromal validity. But, they argue, this does not make the diagnostic constructs useless. Validity is present or not, whereas utility is a matter of degree, and can exist even if validity does not. For purposes of treatment and ongoing research they earn their place in the diagnostic manual as “useful” if not valid categories.
Given that they do not countenance notions such as partial validity or degrees of validity, Kendell and Jablensky would not be in agreement with the idea of weak validity to be described in the next section.
In developing DSM-5, the Work Groups followed the “Guidelines for Making Changes to DSM-V,” most recently revised in 2009 (Kendler et al. 2009). These guidelines may be viewed as an elaboration of the Robins/Guze criteria and certainly represent an excellent, state-of-the-art approach to achieving diagnostic validity with syndromal diagnoses. While Kendell and Jablensky use only the Robins/Guze criteria of laboratory studies and separation from other disorders, the Guidelines use all of the Robins/Guze criteria, plus another proposed by Kendler, “response to treatment” (1990). (For both, the criterion of laboratory studies is an empty set for most of the DSM-5 diagnoses.) The authors begin by stating that the desired paradigm shift and validity won’t be achieved in DSM-5: “While DSM-V will not in itself represent a ‘paradigm shift’, it is intended to start a process that will lead to more useful ways of classifying and diagnosing mental disorders based on our current knowledge and reasoned predictions of where the science is heading” (2009: 1). They outline a list of validators, grouped into categories of antecedent, concurrent, and predictive. They provide careful guidelines for use of the validators in the introduction of new diagnoses, the deletion of existing diagnoses, and any changes in current criteria sets, as well as changes in the relational structuring of diagnoses.
It is important to bear in mind that, in the ongoing absence of definitive findings from basic science that would move a diagnosis toward something more scientifically basic than a symptom cluster, the putative validity of psychiatric diagnoses under the Guidelines remains that of syndromal validity. All questions regarding the validity of a particular diagnosis address the evidence in support (or not) of a particular syndrome. For instance, the high concordance rate of schizophrenia in identical twins lends support to the syndromal validity of schizophrenia as a diagnostic construct.
In discussing syndromal validity in the Guidelines approach, it is important to bear in mind that, in contrast to the “strong” Kendell/Jablensky syndromal criterion of separation from other disorders that lends itself to an either/or, valid-or-not-valid approach, the “weak” syndromal validity of the Guidelines approach implies partial validity, or degrees of validity. To take the example of identical twins, the higher the identical twin concordance rate for, say schizophrenia, the stronger the partial syndromal validity for that disorder.
With the publication of the DSM-5, we know the results of Work Group decisions, although, regretfully, we don’t know the details of discussions that led to the decisions. New diagnoses in the manual include disruptive mood dysregulation disorder and binge eating disorder. Reorganization of existing DSM-IV diagnoses led to combining autism and Asperger’s disorder into a single diagnosis called autism spectrum disorder. Proposed disorders needing further work and listed in Section III include attenuated psychosis disorder and mixed anxiety/depression disorder. As indicated, these changes were all made on the basis of syndromal validity. None of the changes is grounded in neurobiologic evidence.
The strength of the Guidelines approach to syndromal validity is the use of updated validators and the incorporation of the huge body of scientific research carried out since the publication of DSM-TR in 2000, both of which result in a diagnostic manual with much more promise of syndromal validity than any of its predecessors. The weakness of the approach is that DSM-5 may leave us more than not in the same place as DSM-III—diagnoses with some reliability but minimal hope of achieving real validity with the new set of validators. We can recall the words of Kupfer et al. in 2002: “All these limitations in the current diagnostic paradigm suggest that research exclusively focused on refining the DSM-defined syndromes may never be successful in uncovering their underlying etiologies” (p. xx).
To illustrate the weakness, let us consider a couple of major DSM categories, schizophrenia and bipolar I disorder, and test them against the new validators. In the set of three predictive validators (all marked as “high priority”)—diagnostic stability, course of illness, and response to treatment—schizophrenia and bipolar disorder score weakly on all three. The tendency toward diagnostic switches between schizophrenia, schizoaffective disorder, and bipolar disorder undermines the validator of diagnostic stability (Riecher-Rössler and Rössler 1998; Kupfer 2002). The varied course for both disorders challenges the validator of course of illness, and finally, they both show a varied response to treatment, including responsiveness to the same medications. The other validator marked as high priority, familial aggregation and/or co-aggregation, produces a confusing result because with each disorder there is clear familial aggregation, but also aggregation with other disorders. Finally, the validator called biological markers, e.g., molecular genetics and neural substrates, again produces confusing results, inasmuch as hundreds of genetic polymorphisms are involved in these disorders and specific neural substrates have not been confirmed (Craddock et al. 2005, 2006; Hyman 2011.
It is of interest that in 1990 Kenneth Kendler, who was the lead author of the Guidelines report, wrote presciently about the problems in formulating a scientific basis for syndromal diagnoses in psychiatry. He pointed out that the choice of validators and of diagnostic criteria cannot be determined entirely on an empirical basis. The validity of one diagnostic construct versus another will depend on which validators are prioritized. Writing about competing constructs of schizotypal personality disorder, he wrote: “Moreover, empirical data cannot determine which of the two is better, for this decision is essentially a value judgment as to which construct of the disorder is more conceptually appealing” (1990: 971). And more generally, he wrote:
Many important nosology questions in psychiatry are fundamentally nonempirical . . . In many cases, obtaining answers to the little questions will not provide unambiguous answers to the big questions. Before the answers to the little questions can be applied to the big questions, there is an important intervening step. This step often requires answers to other sorts of questions that are usually nonempirical (eg, what are the most important validators for this diagnosis? What are the criteria for defining disorder Q to be a subtype of disorder B? How do we distinguish an Axis I disorder from a Axis II disorder? (1990: 972)
Thus Kendler informed us over 20 years before DSM-5 that syndromal diagnoses within the Robins/Guze framework have their limitations, the main ones being that such syndromal validity is weak, partial, and something of a moving target.
In concluding this section, two remarks are in order. The first addresses the conflict between the two types of syndromal validity. In what I am naming “strong syndromal validity,” Kendell and Jablensky argue for one standard (or validator)—zones of rarity (in Robins/Guze terms, clear separation from other disorders). By this standard, virtually all DSM diagnoses are syndromally invalid.
In contrast, the implicit argument of the Guidelines, using the other Robins/Guze criteria, is that validity is not an all-or-nothing phenomenon, and that we can think in terms of partial validity or degrees of validity. (I have used the term “weak syndromal validity,” but that term, as indicated earlier, implies concepts like partial validity and degrees of validity.) There is a resolution of sorts in this conflict in that the weak or partial validity of the Work Groups is covering much the same ground as the “syndromal utility” of Kendell and Jablensky.
In this disagreement between the two types of syndromal validity, there is probably no empirical way to determine which position is correct. The Kendell/Jablensky approach will leave the DSM diagnoses in a validity-less, never-never land. The DSM-5 Guidelines approach will allow us at least weak, partial validity, but a validity that is unstable and shifting. And of course, this dispute can be restated in terms of which Robins/Guze criteria will be prioritized, a question that does not readily admit of an empirical answer.
While the DSM-5 Work Groups were working on their revisions of the DSM diagnoses, a group at the NIMH were developing a unique approach to etiology, the NIMH Research Domain Criteria project (RDoC), that could have an impact on validity (Cuthbert and Insel 2010; Insel and Cuthbert 2009; Insel et al. 2010). Indeed, less than two years before the publication of DSM-5, Kupfer and Regier (2011) published an article in which, recognizing the looming failure of DSM-5 to fulfill its promises, they attempted to link the DSM-5 project to the newly developing NIMH RDoC project. The DSM-5 leaders expressed a hope that DSM-5 might achieve more validity, and the sought-after paradigm shift, through the accomplishments of the RDoC project.
The RDoC project focuses on neural circuitry as the target area of research, with a goal of correlating specific, observable malfunctions of cognitive and psychological functioning with particular disturbances of neural circuitry. In the preliminary working draft the NIMH scientists have identified five major domains of functioning, each containing a number of more specific constructs. In the RDoC matrix, these domains of functioning are represented as rows, while the units of analysis are represented as columns. Neural circuitry occupies one column as a unit of analysis, although the most important in this research project. Other units of analysis, represented in other columns, are genes, molecules, cells, physiology, behavior, and self-reports. (Please see Table 10.1.)
Table 10.1 Research domain criteria matrix
Reproduced from NIMH Research Domain Criteria (RDoC), Draft 3.1: June 2011 <http://www.nimh.nih.gov/research-priorities/rdoc/nimh-research-domain-criteria-rdoc.shtml> (accessed May 7, 2014).
The goal of the project is to produce fundamental, scientific validity through the identification of specific disturbances in neural circuitry associated with discrete areas of psychological and cognitive malfunction. In the language of genetics these discrete areas of psychological and cognitive malfunction can be viewed as endophenotypes, in contrast to the DSM diagnostic syndromes, which are phenotypes. The RDoC scientists hope to hand the nosologists of the future a list of these endophenotype/neural circuitry correlations, as well as findings from genetics, physiology, and so forth. How these research findings will be assembled into a diagnostic system remains to be seen. The goal is certainly to discover specific neural circuitry disturbances that would represent specific biomarkers with scientific validity for themselves and whatever larger diagnostic scheme they might be part of.
In describing types of validity we can thus include RDoC validity along with strong syndromal (SSV) and weak syndromal (WSV) validity. We should note, however, that RDoC validity is really a form of the strong diagnostic validity described by Kendell and Jablensky as based on defining fundamental characteristics—and not the Kendell/Jablensky strong syndromal validity defined by separation from other disorders. In the Robins/Guze framework, RDoC validity fulfills the criteria of “laboratory studies” that tends to trump the other, syndrome-oriented Robins/Guze criteria. What does, however, differentiate RDoC from the fundamental validity described by Kendell and Jablensky is that, while the latter is focused on validity of diagnostic constructs or categories, the RDoC project focuses on signs and symptoms, and tries to correlate these discrete elements of cognition, emotion, and behavior with discrete disturbances of neural circuitry.
In this discussion of diagnostic validity we must, finally, recognize that the NIMH project represents a huge promissory note. Current scientific support for the RDoC approach is minimal. Recent history of psychiatric nosology is replete with unredeemed promissory notes, and it’s not obvious that the RDoC project will fare better than its precursors. With this hopeful but guarded attitude toward the project, we remain focused on the WSV approach to diagnosis, as that is the mode in which contemporary psychiatry works.
Relevant to all forms of diagnostic validity thus far discussed—SSV, WSV, and RDoC validity—is the issue of complexity. The discussion of validity has proceeded as if complexity were not part of the discussion. In fact, it is, and I will try to explain the role of complexity in this section.
Among the various accounts of complexity (Nicolis et al. 1989; Mitchell 2003, I will focus on the straightforward presentation by Bechtel and Richardson (2010), who describe three levels of complexity: aggregative, component, and integrative.
An aggregative system is purely mechanical, as exemplified by a clock. A clock can be taken apart and reassembled. Each component part can be examined as a separate entity that, together with the other parts, produces the functioning mechanism, the clock. The parts of a clock have no relation to each other except as pieces of a mechanism that, placed together in the proper manner, produce a functioning clock.
Biologic entities cannot be analyzed in this simple, mechanical manner, as they involve higher levels of complexity. What is needed, say Bechtel and Richardson, are two additional levels of complexity in which a particular part cannot be analyzed in isolation from other parts. What a part is and how it works is determined by its relation to the other parts and to the whole system. The whole is understood in terms of the parts, but the parts are also understood in terms of each other and of the whole.
In the simpler form of complex system, termed component system, the parts can still be studied independently, despite the fact that their actual functioning will depend on the total organization of the system. In a more complex system, termed integrated system, the parts lose their independence and can only be studied as components of the integrated system. What they are and how they work will change with the changing organization of the whole system.
I suggest that psychiatric disorders, like the individuals who suffer from them, cannot be analyzed like clocks. They are disorders of complex, biologic systems. It might be debated whether, in the terminology of Bechtel and Richardson, they are component or integrated systems; but they are certainly complex.
We can further relate the complexity of psychiatric disorders to questions of etiology, diagnosis, and validity. First, etiologic factors interact with one another in a complex manner. Second, this complex etiology can produce heterogeneous presentations and frequent comorbidities. Finally, the complexity of the disorders and their etiologies influences our efforts to classify them, and that inevitably implicates validity.
To follow this line of reasoning, let us take the example of schizophrenia. One of the differences in the larger picture of schizophrenia is bad and good outcome. In the terminology of complexity these divergent presentations are the “wholes” that need to be understood in terms of their “parts,” factors such as genes. We know that the genetic underpinnings of schizophrenia are very complicated, with multiple sites and multiple genetic variants. As Hyman wrote, “Robins and Guze developed their view of relatively simple categorical diagnoses at a time when this dizzying level of complexity [in genetics] was barely imagined” (2011: 9). Thus we have to determine how the genetic variants combine to play a role in the heterogeneous presentations of schizophrenia. But we also have to pay attention to how epigenetic factors affect genetic expression—that is, how environmental factors may determine whether a relevant gene is expressed or not. Thus, in analyzing, say, how good outcome schizophrenia (the “whole”) is determined by the shared genetic “parts” of this group, we will also have to recognize that elements of the “whole” (e.g., psychological motivation, behavior, treatment, parental relationship, socioeconomic setting) are affecting how the genetic “parts” are expressed. We now have a situation in which the schizophrenic person is strongly affected by his/her genetic constitution but at the same time affects the genes that are affecting him. We cannot evaluate an individual’s genetic constitution apart from understanding how that genetic structure is working in that individual. In the case of a shared presentation, e.g., “good outcome,” we will not only look for a shared genetic picture (the part) but also a shared epigenetic/phenotypic picture (the whole). In this example the good outcome would be both a result of the genetic structure and a cause of the genetic structure, and the genetic structure would be a cause of the good outcome presentation and a result of it.
Let me further illustrate complexity in psychiatric diagnosis with two other examples. The first is a recent article by Kendler (2012) in which he analyzes the respective causal variance of schizophrenia, major depression, and alcohol dependence, all displayed in elegant column charts. (Please see Figures 10.1, 10.2, and 10.3). According to Kendler, each of these causal factors is a difference maker, meaning that when an outcome is the result of a variety of contributing factors, a difference maker (or risk factor) is one whose presence or absence changes the probability of the outcome. For each diagnosis the causal variance adds up to 100 percent. Kendler recognizes that he is adding the causal factors in a simple, aggregative manner, as if each is acting independently of the others; and that this presents a false picture of factors that interact with one another. He writes: “. . . for pragmatic reasons, I initially assume an independence of difference-makers that does not exist in nature” (2012: 379). And at the end of the article he concludes:
The results of the empirically based pluralistic analysis of the causes of SZ, MD and AD reinforce the conclusions from a prior essay that the commonly expressed wish to develop an etiologically based nosology for psychiatric disorders is deeply problematic. Psychiatric disorders are a result of multiple etiological processes impacting on many different levels and often further intertwined by mediational and moderational interactions between levels. It is not possible a priori to identify one privileged level that can unambiguously be used as the basis for developing a nosologic system. (2012: 385)
Fig. 10.1 Schizophrenia. Reprinted by permission from Macmillan Publishers Ltd: Molecular Psychiatry, 17(4), The dappled nature of causes of psychiatric illness: replacing the organic–functional/hardware–software dichotomy with empirically based pluralism, pp. 377–88, doi:10.1038/mp.2011.182 (c) 2012, Nature Publishing Group.
Fig. 10.2 Major depression. Reprinted by permission from Macmillan Publishers Ltd: Molecular Psychiatry, 17(4), The dappled nature of causes of psychiatric illness: replacing the organic–functional/hardware–software dichotomy with empirically based pluralism, pp. 377–88, doi:10.1038/mp.2011.182 (c) 2012, Nature Publishing Group.
Fig. 10.3 Alcohol dependence. Reprinted by permission from Macmillan Publishers Ltd: Molecular Psychiatry, 17(4), The dappled nature of causes of psychiatric illness: replacing the organic–functional/hardware–software dichotomy with empirically based pluralism, pp. 377–88, doi:10.1038/mp.2011.182 (c) 2012, Nature Publishing Group.
This study is focused on etiological complexity, but, as just noted, etiological complexity complicates our efforts to develop and validate classifications. For instance, if causal factor x is of a certain strength, but is also affected by other causal factors, and also judged non-empirically to be highly relevant in relation to other factors, how should we combine all this information to make a summary judgment about the “validity” of the diagnostic construct in question?
The second example is the RDoC matrix, as discussed earlier. The matrix describes domains of function as rows, and units of analysis as columns. The cells represent areas of analysis, where a particular subdomain is matched against a particular unit of analysis, e.g., neural circuitry. In its graphical presentation the matrix might look as simple as a clock. But it is not, and that’s where the complexity comes in. Imagine taking the subdomain attention and matching it against the unit of analysis circuits. The area of study for that cell would be what kind of dysfunction in neural circuitry could be associated with a dysfunction in attention. But there are several other columns for studying the dysfunction of attention. Under genes, we would study genetic patterns associated with the defect in attention, and under behavior we would study the contribution of disturbed behavioral patterns to problems with attention.
Each of the units of analysis could themselves be complex. Under genes, for example, the goal would be a straightforward genetic pattern associated with the endophenotype/attentional dysfunction, but in reality we can expect to find several genetic polymorphisms, all affected by epigenetic factors, converging on the deficit in attention. And of course the factors from each of the units of analysis—circuits, genes, behavior—may all affect one another, e.g., behavior affecting how genes are expressed. The conclusion is that emphasizing a single unit of analysis (column) is ultimately not a realistic model, since for a particular subdomain of dysfunction the respective units of analysis (columns) will interact with one another, and in various combinations affect the emergence of the deficit. And finally, to make this analysis even more complicated, we should recognize that the various domain (and subdomain) units (the rows) may be interacting and affecting one another in ways that we don’t yet understand. These two examples of etiologic complexity illustrate how such complexity leads to a rather polymorphous view of nosology and validity. We have seen earlier in the chapter how the prioritizing of one validator over another can lead to the choice of one diagnostic construct over another (Kendler 1990). If we now add etiologic complexity, we can see how emphasis on one or another view of etiologic relationships could alter both validators and constructs. Etiological complexity thus worsens the validator and construct ambiguity already highlighted by Kendler in 1990, along with the inevitability of non-empirical decisions in deciding the issues in question.
The “official” way of classifying in psychiatry is biomedical. Psychiatric disorders and diagnoses are expected to follow the model of the rest of medicine, with psychiatric disorders and diagnoses rooted in biomedical pathology. Ideally, the model would achieve discrete, medically founded, valid diagnoses.
Inasmuch as all forms of validity thus far discussed— strong syndromal validity (SSV), weak syndromal validity (WSV), RDoC, and other fundamental validity—work within the biomedical model, we have taken the model for granted. But are there in fact limitations in the model that bear on the question of diagnostic validity? Keep in mind that biomedical validity for the DSMs has meant adherence to the Robins/Guze standards of validity. Thus questioning limitations in the biomedical model is the same as questioning limitations in the Robins/Guze standards.
The limitation often noted is that many clinical presentations don’t fit neatly into one of the biomedically oriented DSM diagnoses, creating problems of NOS diagnoses (now relabeled “unspecified,” following the language of ICD-10 and ICD-11), excessive comorbidity, and failed separation from other disorders. From the biomedical perspective, such problems require only further refinement of the biomedical categories and do not represent any inadequacy of the model itself.
The limitation of the biomedical model I want to focus on in this section pertains to psychology. Psychological problems are the elephant in the room of biomedical nosology. The notion that there might be a significant psychological problem or conflict with resolution through some kind of psychological intervention has no place in biopsychiatry. The biomedical message tends to be, please leave your inner life at the door. Indeed, we could say that the DSM is designed for medically oriented psychiatrists. The rest—clinical psychologists, clinical social workers, psychotherapeutically oriented psychiatrists—live with the manual as they must, but for many it does not reflect their way of understanding their patients nor their way of carrying out their clinical work.
Of course, defenders of the biomedical model would disagree with this statement. To cite one of biomedical psychiatry’s pioneers, Samuel Guze articulated the biomedical attitude toward psychological etiology and treatment, arguing that a psychological dimension of treatment may be important as supportive therapy, but that psychological factors are neither causative nor curative (Guze 1989).
To begin a discussion of psychology and the DSM, let us keep in mind that the Robins/Guze criteria and the DSMs have always included psychological symptoms, and that the diagnostic criteria in great part describe such symptoms. But how to understand these symptoms? In a biomedical framework these “mental” or psychological symptoms always reflect an underlying biomedical condition, i.e., the “real” parts. We can question, however, whether symptoms are at times more than surface manifestations of underlying biomedical processes. Take borderline personality disorder (BPD). We have DSM diagnostic criteria describing behaviors such as “unstable and intense interpersonal relationships,” feelings such as “emptiness,” and the psychologically complex criterion of “identity disturbance.” Is identity disturbance merely a surface symptom reflecting an underlying biomedical disorder? Is it not, rather, an indicator of a complicated psychological process—both a symptom and part of the process. To the biomedical psychiatrist who insists that identity disturbance is only a symptom in the sense of a surface manifestation, we might respond, pace Guze and others, that biomedical treatments don’t work very well for BPD, and that effective treatment is psychological.
To understand the role of psychological factors in psychiatric disorders, their etiologies, and their treatments, we can reflect back to the previous section on complexity. To stay with the example of BPD, there are multiple etiological factors—biological, psychological, social, etc.—and, as suggested by Kendler in the article previously cited (2012), you can’t simply add them up in an aggregative manner. Each affects the etiological power of the others. For BPD the psychological factor (e.g., early trauma) may be strong, but that factor will be affected by biological factors like constitutionally determined sensitivity to stress. The same complexity applies to treatment. For BPD the most effective treatment may be psychological, but for some patients medications may be very helpful, and in that situation the two forms of treatment interact in a complex manner, the medication supporting the psychotherapy and vice versa.
Let me illustrate these points with a case example. Linda, a single woman in her early forties, presented to me with symptoms that clearly warranted a diagnosis of PTSD. She did, however, also meet criteria for generalized disorder and borderline personality disorder. Her array of symptoms was likely related to early sexual abuse, of which she was only minimally aware at the onset of treatment, as well as to a non-supportive parental background. The patient was educated as a medical doctor, but with the burden of intrusive flashbacks and anxiety symptoms, she found it very difficult to sustain the responsibilities of practicing as a physician, and would for periods of time stop working as a physician and do other medical work for which she was qualified.
The same problems encumbered her personal life. Issues of anxiety, distrust, and self-doubt prevented her from forming the kind of stable, intimate relationship she desired.
The treatment has been primarily psychotherapy, focused both on the early abuse and the current symptoms and life problems. Medication has not been very helpful and has been only a minimal aspect of the treatment. She has been in treatment for about three years and has made slow, steady progress; she spends more time working as a physician, and she tolerates better the anxiety of being in a relationship.
This woman illustrates the complexity of etiology, treatment, diagnosis, and psychology in psychiatry. Etiology includes the interweaving combination of early sexual abuse, poor parental support, and, probably, an inborn tendency toward heightened anxiety. Treatment has been primarily psychological, with minimal help from medications. Diagnosis is confusing, as indicated earlier. Of the three DSM diagnoses mentioned, we could prioritize one (e.g., PTSD) and name the other two as comorbid diagnoses. Or we could pick one diagnosis and argue that all the symptoms could be included within that diagnosis. Here we face the dilemma described earlier of making diagnostic decisions on a non-empirical basis. Finally, we are dealing with a case in which psychological factors play a significant role—in etiology, presentation, and treatment. Regarding presentation, we recognize psychological symptoms as well as psychological conflicts. The latter are multiple: conflicts over accepting the reality of the abuse, over facing her parents’ failures, over tolerating the anxiety and other symptoms when working as a physician and developing a relationship, over transforming her self-image from impaired victim to competent adult, and of course over accepting the treatment and the treater as helpful rather than as causing more symptoms. We have worked on all these conflicts with understandably slow progress.
I invoke this patient as someone whose problems involve significant psychological factors in etiology, presentation, and treatment. Guze and others would not countenance as a psychiatric disorder a case where etiology and treatment are to a significant degree psychological, as opposed to “biomedical.” The irony of this case is that the patient does fit more than one of the standard DSM diagnoses. It’s just that the respective diagnoses have to be understood more complexly than the strictly biomedical model would allow.
To conclude this section, let me summarize the argument. Psychological etiology and psychological treatment exist on a continuum in psychiatric disorders. At one end of the spectrum, e.g., schizophrenia, psychological etiology is minimal and psychological treatment is supportive and rehabilitative. In the middle of the continuum are the many psychiatric disorders, e.g., depressive disorders, in which the degree of psychological etiology varies from one patient to another, and psychological treatment is supportive and at times may include insight-oriented psychotherapy. At the other end of the continuum, as with Linda, psychological etiology is stronger, and psychological treatment may be primary.
As the psychological dimensions of etiology and treatment become stronger, they challenge both the biomedical model and the diagnostic validity based on it. We discuss both in the following section.
The discussion of validity in this chapter has taken the following course. I began with the failure of the DSMs (including DSM-5) to meet the Robins/Guze standards of diagnostic validity in psychiatry. From there we moved to a distinction between strong (SSV) and weak (WSV) syndromal validity, the first, as developed by Kendell and Jablensky, highlighting the Robins/Guze criterion of separation from other disorders as the single standard for strong syndromal validity, the second, as developed in the DSM Guidelines, highlighting the other Robins/Guze criteria for partial, weak validity (the Robins/Guze criterion of “laboratory studies” has played a minimum role for both Kendell and Jablensky and the Guidelines, since for most DSM diagnoses such studies have not yielded anything like a definitive biomarker).
Given the inability of DSM nosology to achieve SSV, we have been left with WSV as the current standard of syndromal validity. A major consequence of this standard is that syndromal validity is vulnerable to the non-empirical prioritizing of one validator or diagnostic construct over another, thus rendering validity a moving target.
As a next step in the argument I introduced the RDoC project, which promises to bypass syndromal validity and achieve the Robins/Guze criterion of “laboratory studies” (described by Kendell and Jablensky as “fundamental validity”). Given that the RDoC project remains a promissory note, it is not in a position at this time to replace WSV as the standard of validity.
I next introduced the concept of complexity in psychiatric diagnosis. Although primarily focused on the complex etiologies of psychiatric disorders, complexity inevitably involves presentation, diagnosis, and validity. The effect of recognizing complexity in the diagnostic process is that it renders weak syndromal validity even more complicated (e.g., validators interacting with one another), and still more vulnerable to shifting conclusions about diagnostic validity.
Finally, I took the question of complexity a step further in pointing out that the preceding discussion of diagnostic validity, developed in terms of the Robins/Guze criteria, had been framed within the biomedical model assumed by Robins and Guze. I questioned the limitations of that model, especially its resistance to including psychological phenomena in the etiology, nosology, and treatment of psychiatric illness. Inasmuch as the preceding discussion of diagnostic validity was predicated on an assumption of the biomedical model, any challenge to that model would involve consequences for our notions of validity.
Let us now follow the next steps in this train of thought. We have seen two problems with diagnostic validity in psychiatry. The first is that the prevailing standard of weak diagnostic validity (WSV) involves non-empirical decisions about which validators to prioritize, and thus what is to count as a valid diagnostic construct. The second is that the biomedically oriented DSMs allow insufficient attention to psychological dimensions of etiology, nosology, treatment, and validity.
How might we address these two problems, and perhaps relate them? Since WSV is already by necessity flexible in its attitude toward diagnostic constructs and validity, is there any reason not to make it still more flexible and include the missing psychological aspects in WDV diagnoses? Allowing such phenomena into WSV nosology, of course, would require a framework larger than the Robins/Guze biomedical framework—or perhaps an expansion of what a biomedical model is.
Let me focus on the latter possibility, proposing that we do not need another model, biopsychosocial or whatever, so much as a complete biomedical model that understands both the complexity of psychiatric disorders and the fact that such complexity includes psychological dimensions not included in the Robins/Guze biomedical framework. In that regard, we have already noted that the diagnostic criteria for some DSM-5 diagnoses contain psychological symptoms that are more than simple markers for underlying biomedical diseases, and that such criteria and symptoms suggest psychological problems and potential psychological treatment. Thus, for diagnoses like BPD and PTSD, DSM-IV had already expanded beyond what would be allowed in strict adherence to the Robins/Guze biomedical framework. Regarding treatment, a complete biomedical model would recognize that there is not biomedical treatment and psychological treatment; there is biomedical treatment that is psychological as well as “medical.”
The philosophical point of this proposed revision of the biomedical model is that subjective or mental symptoms are not always surface manifestations of an underlying biomedical disease process; they may also be the surface manifestations of an underlying psychological process. What makes psychiatry complex is that symptoms are often both, as in many depressive disorders. Psychological problems do not exist without their biology, and biological conditions do not exist without their psychology.
When Kupfer et al. lamented in 2002 that validity would not be achieved with the current Robins/Guze DSM paradigm and would require a new paradigm, they certainly did not have in mind the mini-shift suggested here. The latter is simply a recognition that a syndromal diagnosis could be something more than a symptom cluster representing underlying biomedical disease, and that an expanded biomedical model could include psychological dimensions within WSV.
Regarding my proposal and the structure of the DSM manual, we could imagine creating new, psychologically oriented diagnoses, or simply reformulating the existing sets of diagnostic criteria to include psychological factors. As indicated earlier, the current diagnostic criteria for BPD already describe psychological symptoms that at least suggest underlying psychological disturbance. On the other hand, the diagnostic criteria for major depression present psychological symptoms that seemingly assume underlying biomedical illness. The controversy over the grief exclusion in DSM-IV—and its removal in DSM-5—suggests a reluctance to specify psychological factors in the underlying disorder.
Finally, any restructuring of the diagnostic constructs and diagnostic criteria to include a richer psychological aspect would have some effect on diagnostic validity. In the context of weak syndromal validity (WSV), the diagnostic constructs, already complex, would be even more complex.
The index of DSM-5 does not have a single entry for the word “biomedical.” Use of that term in this chapter is then not warranted by its presence in the manual. Rather, use of the term is justified by the fact that the assumptions of the Robins/Guze biomedical model are so deeply embedded in the DSMs that they don’t need to be mentioned. The goals of psychiatric research remain, as indicated by Kupfer et al. at the beginning of the DSM-5 era (2002), to fill in the Robins/Guze “standard of ‘laboratory studies’” with neuroscientific findings and the infamous biomarkers, replacing WSV not with SSV, but with fundamental validity. In this chapter I have proposed an inclusion within WSV of psychological factors, and with that an expansion of the biomedical model beyond the narrow Robins/Guze standards. Such a change would leave the DSM in the territory of WSV, and would do nothing to achieve fundamental validity. What it would do is make the DSM more useful for psychologically oriented clinicians.
Meanwhile, the architects of the RDoC project, in their quest for fundamental validity, define psychiatric disorders as “brain disorder,” and attempt to locate the disorders in neural circuitry. In this RDoC world psychology would again be minimized. The RDoC matrix does have an analysis/column marked “psychological,” but it’s unclear what that might mean other than psychological, surface manifestations of the disturbed circuits—thus no different from the traditional biomedical framework as described earlier. The challenge in this new/old order of brain diseases would be to again find a place for psychology in the diagnostic framework.
American Psychiatric Association (1980). Diagnostic and Statistical Manual of Mental Disorders (Third Edition). Washington, DC: American Psychiatric Association Press.
Bechtel, W. R., and Richardson, C. (2010). Discovering Complexity. Decomposition and Localization as Strategies in Scientific Research. Cambridge, MA: MIT Press.
Clarke, D., Narrow, W., Regier, D., et al. (2013). DSM-5 field trials in the United States and Canada, Part I: Study design, sampling strategy, implementation, and analytic approaches. American Journal of Psychiatry, 170, 43–58.
Craddock, N., O’Donovan, M., and Owen, M. (2005). The genetics of schizophrenia and bipolar disorder: dissecting psychosis. Journal of Medical Genetics, 42, 193–204.
Craddock, N., O’Donovan, M., and Owen, M. (2006). Genes for schizophrenia and bipolar disorder? Implications for psychiatric nosology. Schizophrenia Bulletin, 32, 9–16.
Cuthbert, B., and Insel, T. (2010). The data of diagnosis: New approaches to psychiatric classification. Psychiatry, 73, 311–14.
Cuthbert, B., and Insel, T. (2010). Toward new approaches to psychotic disorders: The NIMH Research Domain Criteria project. Schizophrenia Bulletin, 36, 1061–2.
Detre, T. (1987). The future of psychiatry. American Journal of Psychiatry, 144, 621–5.
Feighner, J., Robins, E., Guze, S., et al. (1972). Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry, 26, 57–63.
Guze, S. (1989). Biological psychiatry: Is there any other kind? Psychological Medicine, 19, 315–23.
Hyman, S. (2011). Diagnosis of mental disorders in light of modern genetics. In D. Regier, W. Narrow, E. Kuhl, and D. Kupfer (eds), The Conceptual Evolution of DSM-5. Washington, DC: American Psychiatric Publishing, 3–18.
Insel, T., and Cuthbert, B. (2009). Endophenotypes: bridging genomic complexity and disorder heterogeneity. Biological Psychiatry, 66, 988–9.
Insel, T., Cuthbert, B., Garvey, M., et al. (2010). Research Domain Criteria (RDoC): Toward a new classification framework for research on mental disorders. American Journal of Psychiatry, 167, 748–51.
Kendell, R., and Jablensky, A. (2003). Distinguishing between the validity and utility of psychiatric diagnoses. American Journal of Psychiatry, 160, 4–12.
Kendler, K. (1990). Toward a scientific psychiatric nosology: strengths and limitations. Archives of General Psychiatry, 47, 969–73.
Kendler, K. (2012). The dappled nature of causes of psychiatric illness: Replacing the organic–functional/hardware–software dichotomy with empirically based pluralism. Molecular Biology, 17, 377–88.
Kendler, K., Kupfer, D., Narrow, W., et al. (2009). Guidelines for Making Changes to DSM-V Revised 10/21/09. Available at: <http://www.dsm5.org/ProgressReports/Documents/Guidelines-for-Making-Changes-to-DSM_1.pdf>, (accessed January 22, 2014).
Kupfer, D., First, M., and Regier, D. (eds) (2002). A Research Agenda for DSM-V. Washington, DC: American Psychiatric Association Press.
Kupfer, D., and Kraemer, H. (2012). What we Learned from the DSM-5 Field Trials. Psychiatric News (December 16, 2012).
Kupfer, D., and Regier, D. (2011). Neuroscience, clinical evidence, and the future of psychiatric classification in DSM-5. American Journal of Psychiatry, 168, 1–3.
Mitchell, S. (2003). Biologic Complexity and Integrative Pluralism. Cambridge: Cambridge University Press.
Nicolis, G., Nocolis, G., and Progogine, I. (1989). Exploring Complexity: An Introduction. New York: W. H. Freeman and Company.
Pies, R. (2007). How objective are psychiatric diagnoses? Psychiatry, 4, 18–22.
Regier, D., Narrow, W., Clarke, D., et al. (2013). DSM-5 field trials in the United States and Canada, Part II: Test–retest reliability of selected categorical diagnoses. The American Journal of Psychiatry, 170, 59–70.
Regier, D., Narrow, W., Kuhl, E., et al. (eds) (2011). The Conceptual Evolution of DSM-5. Washington, DC: American Psychiatric Publishing.
Riecher-Rössler, A., and Rössler, W. (1998). The course of schizophrenic psychoses: What do we really know? A selective review from an epidemiological perspective. European Archives of Psychiatry and Clinical Neuroscience, 248, 189–202.
Robins, E., and Guze, S. (1970). Establishment of diagnostic in psychiatric illness: Its application to schizophrenia. American Journal of Psychiatry, 126, 983–987.
Spitzer, R., Endicott, J., and Robins, E. (1978). Research diagnostic criteria: Rationale and reliability. Archives of General Psychiatry, 35, 773–82.