Reality and utility unbound: An argument for dual-track nosologic validation
Adriano C. T. Rodrigues and Claudio E. M. Banzato
Despite recurring claims that psychiatric diagnosis and diagnostic categories in psychiatry should be valid (Kendell 1989; Spitzer 2001; Rounsaville et al. 2002), the very notion of validation is not only multifarious but also an object of terminological and conceptual disagreement (Nelson-Gray 1991; Blashfield and Livesley 1991; Skinner 1981; First et al. 2004; Löwe et al. 2008). Indictments of psychiatric diagnostic categories for not being valid, for example, are brought up under diverse justifications, ranging from the lack of evidence for their reality to their allegedly poor theoretical grounding (Szasz 1960; Trafimow 2010; Skinner 1981).
While different criteria and procedures for the validation of psychiatric diagnostic categories have been proposed and applied (Robins and Guze 1970; Kendler 1980; Andreasen 1995; Skinner 1981; Stoyanov 2009), discussions of competing views are scarce (Zachar and Kendler 2007).
As a response to this scenario of uncertainty and ambiguity, the aim of this chapter is twofold. First, we intend to provide some organization to the zoo of validity concepts by suggesting that all validity concepts in the field of psychiatry fit into one of two domains of validation, recognized and termed by Claire Pouncey (2003), to wit, diagnostic and nosologic.
As a second task, we will consider whether the validity concepts and processes of validation in each of these two domains are distinct or similar, and whether this makes it fruitful or futile to assess each of them for every psychiatric disorder. We will suggest that, under the umbrella of diagnostic validity, the notions of face, content, criterion-oriented, and construct validity are highly redundant, as each of them carries little in addition to the concept of diagnostic validity itself, i.e., that a set of diagnostic criteria leads to proper identification of clinical instances of a psychiatric construct. Special attention, notwithstanding, will be given to the domain of nosologic validity. We will argue that the main validity concepts in this domain, namely, the pragmatic and realistic conceptions of validity, not only carry relevant and specific sorts of information, but that the presence of one does not warrant that the other is also present. Accordingly, we will advocate that they should be independently assessed for each mental disorder.
As a first step in order to provide some organization to the various conceptions of validity and processes of validation, it is worthy to point out the often overlooked fact that they do not apply to diagnostic categories but to propositions about those categories. Accordingly, just as it makes no sense to assert that a rock is valid or invalid, it is meaningless to state that a psychiatric diagnostic category is valid or invalid. On the other hand, any meaningful proposition about a rock or a psychiatric diagnostic category—for example, on the composition of the former or on the utility of the latter—is amenable to having its validity ascertained. Indeed, when some given diagnostic category is said to be valid or invalid, what is actually meant is the validity of some specific proposition about it.
Of course, not all propositions about psychiatric diagnostic categories matter to nosologists and clinicians. The proposition “schizophrenia is poorly portrayed in the movies,” for example, although an amenable target to the validation process, would hardly be considered of major nosologic concern. In order to properly assess the validity of psychiatric classificatory concepts, a critical step is to have a clear idea on what propositions about these objects are relevant in our field. Indeed, having a clear idea on what propositions matter would allow for a classification of the domains on which psychiatric diagnostic categories need to be validated.
Accordingly, although the psychiatric literature alludes to various types of validity, it is possible to sort all of them into the two related but distinct domains insightfully distinguished by Pouncey (2003, p. 9) as the nosologic and the diagnostic.
In the nosologic domain, the processes of validation refer to the proposition (or hypothesis) that a given diagnostic category is reasonable. That is, there should be a good justification for including it in the system.
As regards to what makes a diagnostic category reasonable, the notion of nosologic validity leaves room for different views on that question. Indeed, what Pouncey (2003) generically calls nosologic validity is represented—both in literature and actual programs of validation—by two different conceptions of validity that we call the pragmatic conception and realistic conception (Spitzer 2001; Kendell and Jablensky 2003; Zachar 2010; Pies 2011). At the core of the pragmatic conception is the assumption that a diagnostic category should be taken as a valid kind if it is useful. At the core of the realistic conception is the assumption that a diagnostic category is valid only if it represents a real entity.
The second domain of interest to which the notion of validity arguably applies, the diagnostic domain, largely overlaps with psychometrics and is usually assessed by means of procedures related to the well-known notions of face validity, content validity, concurrent validity, predictive validity, and, especially, construct validity (Spitzer and Williams 1985; Jablensky and Kendell 2002). In diagnostic validity what is at stake is: a) how well the criteria for a category portray the psychiatric construct; and b) how well its diagnostic criteria lead to accurate identification of clinical instances of that construct. For example, it is expected that the diagnostic category named schizophrenia, as presented in psychiatric classificatory systems, is an adequate representation of the hypothetical construct schizophrenia—but not of other psychiatric constructs—and that its diagnostic criteria actually enable us to distinguish between schizophrenia cases and non-cases. Obviously, the notion of diagnostic validity embeds the assumption that the construct schizophrenia and the diagnostic criteria for schizophrenia in DSM or ICD are not the same.
We intend to argue that although interwoven, nosologic validity and diagnostic validity are not redundant, either conceptually or in practical terms. In fact, the set of diagnostic criteria that represent a category in our classificatory systems might eventually lead to an unequivocal distinction between cases and non-cases of a hypothetical construct, while that construct in itself lacks justification as a real entity. While it could be argued that a diagnostic category as such would be a nonsensical tool—after all, it identifies clinical instances of a groundless construct—, one should note that, as a tool, it would be doing precisely what it was conceived to do. In other words, a given diagnostic category may have full diagnostic validity (i.e., validity in psychometric terms) while lacking altogether nosologic validity. Conversely, it is also possible for a diagnostic category to have nosologic validity with poor diagnostic validity, as it may prove to be, for any reason, not translatable into the clinical realm. Since they can be independent, both nosologic validity and diagnostic validity should be assessed for every diagnostic category. As regards the process of assessing diagnostic validity, it is arguably not a mystery because the concepts of validity it subsumes are well known and their methodologies are reasonably well described. Whereas the notions of face validity, content validity, and criterion-oriented validity address, by different means, the suitability with which instruments represent our constructs of interest and then lead to the identification of their instances, the prevailing understanding is that the notion of construct validity (as put forward by Cronbach and Meehl 1955) encompasses all their aptitudes and is the only one capable of performing those tasks thoroughly (Cronbach and Meehl 1955; Loevinger 1957; Messick 1990). Thus, construct validity should be probably taken as the representative par excellence of what Pouncey (2003) termed diagnostic validity.
Indeed, the major disagreements on whether psychiatric diagnostic categories are valid or invalid, as well as the major uncertainties on what would make them valid, are both in the domain of nosologic validity. Thus, it is by scrutinizing the realistic and the pragmatic conception of validity—both of which are essentially driven toward defining whether the psychiatric constructs have a legitimate place in nosology—that the following discussions might be illuminating.
Propositions concerning the utility and reality of psychiatric diagnostic categories are felt to be relevant within the nosologic context and, for that reason alone, deserve to have their validity examined. But knowing whether psychiatric diagnostic categories are useful and representative of real entities is not all that matters. In fact, when classificatory meta-theory is at stake, concerns should center on the legitimacy of the criteria and approaches chosen to validate a diagnostic category. Accordingly, because utility and reality are attributes on the basis of which psychiatric constructs are most often justified—a matter of nosologic validation—, their very competence to do so should be carefully assessed.
Although the particular versions of realism are manifold, the most prevailing realistic conception of validation in psychiatry is naturalistic, being characterized by the following presuppositions:
a) valid diagnostic categories are those that represent real mental disorders;
b) a real mental disorder, in addition to being accepted as a disorder, exists in its own right, in the nature, and not only by convention or human artifice.
Accordingly, the diagnostic category “schizophrenia” is valid only if it represents an actually existing disease-entity. In contrast, an example of a non-real (or artificial) disease-entity is calyniophenia, here defined as the co-occurrence of the following: a) avoidance of incorporating new technologies into one’s daily routine; b) aversion to sunlight which is not justified on the basis of visual discomfort, aesthetic preferences regarding skin tone, or fear of skin cancer; and c) rooting for Barcelona soccer team.
In the scientific realm, the methodologies used to confirm the existence of “natural kinds,” including psychiatric nosologic entities, are characterized by the search for evidence that these are not arbitrary collections of features. Two strategies are usually employed. The first is to demonstrate that the manifestations of the disorder have distinctive features that are not included in the entity’s definition. For example, one must show that individuals with a particular disorder, when compared to non-affected individuals, exhibit different biological characteristics, social and functional adjustment, life course, response to therapeutic interventions, or other nosologically relevant characteristics that are not themselves inbuilt in the description of the disorder itself. The assumption is that if a disorder such as schizophrenia exists only by convention, there would be no reason to infer that individuals thus diagnosed would systematically differ from the rest of the population in any other aspect, except for the mental and behavior features that define it (Mill 2002).
A second strategy is to assess whether the defining characteristics of the entity in question cluster together in a non-random way. The defining characteristics should aggregate in the population according to a pattern different from what should be expected to occur by chance. Taking schizophrenia as an example, evidence of its reality could be provided by the demonstration that, delusions, hallucinations, disorganized thinking, grossly disorganized behavior, and negative symptoms co-occur in the general population more often than would be expected by chance.
Both of these strategies assume a conception of reality in which the various features of the disorder take the patterns they do as a result of causal mechanisms shared between different manifestations of the disorder.
Given the previous sketch of a realistic conception of validity, one may ask what sort of special virtues real diagnostic categories are supposed to have and what justifies expecting that psychiatric diagnostic categories with those virtues should be considered nosologically valid.
First, there is the requirement of non-triviality for psychiatric classificatory concepts. Because psychiatry is a practical field, it is only fair to expect that classificatory concepts have implications. Among other things, it is expected that the diagnosis of a given disorder should be helpful, in a diverse degree, to establish prognosis, guide treatment, and estimate the probability of new cases among family members of the affected individual. As artificial diagnostic concepts are not expected to relate in any distinguishable way with variables external to their defining features, they are not believed to allow the inferences that real entities allow.
Among the variables whose association with a psychiatric diagnostic category would grant their non-triviality, etiopathogenic variables enjoy a special reputation, from both scientific and clinical points of view (Kendell 1989; Andreasen 1995). This is explicitly recognized, for example, in the new definition of mental disorder put forward by the DSM-5, according to which “a mental disorder is a syndrome characterized by clinically significant disturbance in an individual’s cognition, emotion regulation, or behavior that reflects a dysfunction in the psychological, biological, or developmental processes underlying mental functioning” (American Psychiatric Association 2013). While correlations with other sorts of variables have broad practical implications, knowing about the causal processes underlying a mental disorder would, hypothetically, provide the best targets for intervention.
A second reason why real diagnostic categories need to be validated is of an ethical nature. The critical argument runs like this: if psychiatric disorders are not real, then psychiatric diagnoses would be little more than instruments of social control, generators of stigma and social segregation. The claim that disorders were not real was integral to the anti-psychiatry movement (Szasz 1960).
Psychiatry’s response to such claims has been the attempt to validate disorders as manifestation of underlying biological mechanisms. In fact, the search for proof that psychiatric nosologic entities are real is a key feature of some of the most influential programs of validation (Gottesman and Gould 2003; Kupfer and Regier 2011).
Several important objections to the realistic conception of validity may be presented.
The first one concerns the fact that, even if psychiatric disorders are indeed real, their very nature can make their reality impracticable to demonstrate. This is because it may not be feasible to accurately assess the required correlations with external variables. For example, the correlations of chemical elements with external variables can be tested in reasonably controlled environments, whereas mental disorders are inescapably placed in very complex scenarios—individuals’ mental, biological, and social lives. By not being separable from the complex biological and mental lives of their bearers, the correlations with other variables are influenced by an unimaginable myriad of factors, falling short of naturalistic standards required for validating real entities.
Such an effect of peripheral factors may be particularly relevant in situations where one is dealing with a network of weak causal forces or INUS conditions (Mackie 1965; Schaffner 2002)—as is likely the case with both mental disorders themselves and the variables whose correlations with mental disorders are investigated in the quest for their reality.
It could be asked then, what is a realistic conception of validity worth if their disorders, however real they may be, cannot be demonstrated to be so? Of course, one sees validation in realistic terms and questions whether it is fair to require that mental disorders have the same degree of specificity in their correlations with external variables as do chemical elements. The adoption of less strict criteria for the reality of mental disorders, however, would require nosologists to confront the uncertainties about which correlations are important and how strong those correlations need to be to refer to real entities. Would the association found between bipolar disorder and suicide (patients with bipolar disorder commit suicide at a much higher rate than the general population), for example, be a satisfactory kind of evidence? What also are we to make of the fact that an association with suicide is not unique to bipolar disorder? For instance, patients with schizophrenia have a high rate of suicide as well.
There are also caveats to be made regarding the assumption that the validation of psychiatric diagnostic categories according to a realistic perspective would be strategically advantageous in the search for etiology and pathogenic mechanisms. While supposedly the constitutive features of diagnostic categories that represent real entities are more liable to having shared causal mechanisms, it should be noticed that the role of causality in psychiatric context is largely a practical one. As perceptively observed by Kendell (1989, p. 45), there is nothing inherently important about causal elements, except for the openings for intervention they eventually provide. At the same time, etiological and pathogenic knowledge is far from being an imperative for interventions to occur appropriately.
In addition, many effective interventions on mental disorders do not depend on prior knowledge of their causes. In fact, the opposite is the rule—namely, knowledge about pathogenesis often is gained by the discovery of successful interventions.
Although validating a psychiatric diagnostic category according to the realistic conception may facilitate the discovery of their underlying causal mechanisms, and although this knowledge may eventually become useful, the practical purposes of those classificatory concepts can arguably be met without that aid. In fact, it is questionable whether etiologically oriented psychiatric classifications should be pursued with such priority and such high expectations, since, as well stated by Kenneth Schaffner (2002), they are not always the most useful for clinical purposes.
The ethical recommendation that psychiatric diagnostic categories should be valid in realistic terms must also be taken with caution. It is evident that a diagnostic category should never be accepted primarily because it serves private, political, or corporative interests. However, it is disputable that the acceptance of diagnostic categories must be based on evidence that they represent real disease-entities. Particularly in psychiatry, one must treat with skepticism an assertion that a diagnostic category so closely connected to the human condition, to the experience of self, and to intersubjectivity, should have its reality ultimately dependent on its biological nature. While reality and nature are highly valued by the anti-psychiatry movement as validating criteria, this is not because of the intrinsic reach that reality and nature have as validators (e.g., nature as a neutral and fair umpire). Instead, it is a strategy to curb personal influences and class interests in the prescription of normalcy and psychopathology.
It goes without saying that, as regards human experience, other parameters may be more meaningful and more ethically oriented than reality and natural status. The personal significance of the lived experience, for example, may be one of these criteria. This includes the desire to change, to suffer less, and to acquire the kind of flourishing life one sees in others, and to share that with others. These all are relevant subjective criteria that can play a robust and unbiased role in establishing the cartography of psychopathology.
According to the pragmatic conception of validity—elsewhere called information-based, instrumentalist, and utilitarian conceptions of validity (Zachar and Kendler 2010; Rodrigues and Banzato 2009; Pies 2011)—nosologically valid diagnostic categories are those that are useful. It should be noted, however, that usefulness is not unique to the pragmatic conception. Realistic conceptions of validity aim at such a goal too, by searching for correlations that are supposed to make psychiatric diagnostic categories useful tools. In the pragmatic conception of validity, however, the usefulness of classificatory concepts is not coupled with any ontological claim. Usefulness is, in itself, what confers nosologic validity to psychiatric diagnostic categories. Nor does it matter how a diagnostic category happens to be useful, only that it be useful.
Because medicine is an eminently practical field, an important question for a pragmatic conception of validity is not why psychiatric classificatory concepts should be validated on the basis of their usefulness, but why such a conception of validity leaves ontological considerations aside. This is explained for epistemological or methodological reasons. Epistemologically, once the usefulness of a diagnostic category is demonstrated, metaphysical elaboration will neither increase it nor discredit it. Methodologically, not engaging in philosophical theorization may represent a strategic move to taking as much practical and scientific advantage as possible from extant diagnostic categories, even if one hopes that this is a provisional step on the path toward a nosology that is valid in realistic terms. In any case, by not engaging in metaphysical speculation, the intrinsic value of utility as a criterion of validity is emphasized, even in the face of uncertainties on the links between reality and usefulness in science.
But how reasonable is it, after all, to conceive utility as a criterion of validity aside from ontological and epistemological considerations? Is it reasonable to consider utility an independent criterion of validity, assuming that a diagnostic category may be useful without mirroring a real entity? We will explore this question over the next two sections.
Let us consider, first, a scenario in which it is impossible to prove that psychiatric classificatory concepts represent “real” kinds or in which such a hypothesis has been refuted. Would the failure to be a real kind undermine the usefulness of such classificatory concepts and deny them any nosologic validity?
Some insights in this regard can be gained from considering the cases of man-made kinds such as capitalism, poverty, democracy, law, and political affiliation, all of which are pregnant with practical consequences. Whereas lacking existence aside from social interactions, perception, and human ingenuity, as these concepts get progressively enmeshed in culture and become part of the repertoire of concepts people use to deal with the world, they often end up being indispensable for the apprehension of human reality. Strictly speaking, their utility is not necessarily narrow in scope or contingent upon correlations, be they causal, transient, or spurious. In fact, their utility may come from the conceptual networks engendered, which may have a reach similar to the theoretical import of those scientific concepts thought to stand for “real” kinds.
Similarly, it is not necessary to assume that, to be useful, psychiatric diagnostic categories must represent real mental disorders. If this is so, and psychiatric diagnostic categories can fulfill the practical role that medicine asks of them irrespective of their ontological status, utility should indeed be considered a validity criterion in itself.
If a concept cannot have its utility dismissed simply because it does not represent a real entity, would representing a real entity ensure its usefulness? Turning the question around in this way is important in a realistic context because, when it is assumed that valid mental disorders must be real entities, making utility a criterion of validity for psychiatric classificatory concepts is, at best, a provisional measure and, at worst, a deviation off the right track. To review, three related features are most often attributed to real entities in the scientific realm, although variably prioritized: the existence of an underlying causal mechanism, a set of shared properties as a result of shared causes, and a predictable pattern of correlations with external variables (non-triviality). Whichever of these features is given priority, their mutual connections are such that a non-random pattern of correlations between real entities and external variables is always expected, even when not taken as a core aspect of real entities from the beginning. Of course, not all correlations between a putatively real disease entity and external variables matter in the nosologic realm, just those that have actual or potential practical bearing on what we want to use the diagnostic category for. While being associated with preference for the green color makes a diagnostic category non-trivial in some sense (maybe for fashion designers), nosologic relevance would require the category to be correlated with variables that play a role in the field. Highly valued would be correlations with a given genetic allele, or a specific biochemical cascade.
Although considered nosologically relevant, it is an open question as to whether such correlations provide clinical utility. At present, these kinds of correlations have not been proven clinically useful—for various reasons, ranging from the lack of replicability of these findings to the lack of technologies for designing interventions that link bench to bedside. That is, whereas correlations with genetic and biochemical variables (and many others) seem relevant from a theoretical and scientific standpoint, this suggests their potential rather than actual usefulness.
In sum, realistic confirmation of psychiatric classificatory concepts does not imply usefulness (though that is generally expected to be the case), and lack of reality does not imply non-utility. There is an asymmetric relation between the two. While a pragmatic utilitarian stance of validity is ontologically agnostic, the realistic one aims at being useful, even if its utility cannot ultimately be warranted. Therefore, as regards psychiatric classificatory concepts, reality and utility cannot be reduced to one or the other. As a result, the pragmatic conception of validity deserves to stand on its own, free of ontological and epistemological ties, in parallel to the realistic conception of validity.
Both reality and utility are highly valued validators of psychiatric diagnostic categories. As psychiatry’s ultimate purpose is practical (as it is for science as a whole), utility certainly deserves a special place among validators. Any psychiatric nosology ought to be considered a failure if, at the end, it has been proven to be useless for whatever purpose it has. Psychiatric nosology, however, is still in motion, and any diagnostic category taken as promising from the realistic point of view—however of little use it may provisionally be—should be taken as nosologically valid to a certain extent, for all the prospects entailed.
As we have shown, the attributes reality and utility are not redundant and they are not necessarily coupled with each other. Indeed, because there is no fixed standard of relationship between reality and utility, the nosologic significance they both grant to psychiatric diagnostic categories suggests that it is possible and beneficial to run simultaneously projects of validation along realistic and pragmatic lines. As much as continuous epistemic interactions between top-down and bottom-up classificatory approaches have been argued to be advantageous for the nosologic enterprise (Kendler 2009), a similar strategy could perhaps get the most from pragmatic and realist approaches to validation, by means of their mutual enhancement.
In fact, although a pragmatic approach in validation programs prioritizes the search for correlations that are promptly useful, there is no obstacle for such a program to also be considered the initial step of a realistic program. In other words, instrumental categories can be thought of as signposts on the way to a more realistic conception. Similarly, while a realistic approach in validation programs would involve searching for more extensive networks of correlations, eventually turning diagnostic categories into privileged points of theoretical intersection, it could primarily emphasize a subset of variables that fall under a more instrumentalist conception. Arguably, this would amount to a prospective program with increased likelihood of maximizing the utility of their objects of study.
Of course, triangulation is not the only strategy by means of which psychiatry could take as much advantage as possible from the virtues of pragmatic and realistic approaches. Provided that they will not overshadow each other, both programs of validation could be left free to follow their own paths in parallel, with no pre-established degree of interaction, up to the point that one of them prevails or they are shown to be inescapably complementary. Even different classificatory systems could be held at a given time, for different purposes, if the most clinically useful way to classify mental disorders does not happen to be also the most promising from a scientific point of view. There is no way to know in advance which the best path to follow is or how this story will end.
What is put forward here, evidently, is not a new or separate conception of validity. At most, it should be seen as a conceptual framework for considering the legitimacy of competing validation criteria in psychiatric nosology. The position presented here favors the combined and context-sensitive use of pragmatic and realistic approaches. Thus, the contours of the validation programs will always depend on the developments in the field and on the newly emerging interests, which we are not in a position to anticipate.
American Psychiatric Association (2013). Diagnostic and Statistical Manual of Mental Disorders—Fifth edition (DSM-5). Washington: American Psychiatric Publishing.
Andreasen, N. C. (1995). The validation of psychiatric diagnosis: new models and approaches. American Journal of Psychiatry, 152(2), 161–2.
Blashfield, R. K., and Livesley, W. J. (1991). Metaphorical analysis of psychiatric classification as a psychological test. Journal of Abnormal Psychology, 100(3), 262–70.
Cronbach, L., and Meehl, P. E. (1955). Construct validity of psychological tests. Psychological Bulletin, 52(4), 281–302.
First, M. B. et al. (2004). Clinical utility as a criterion for revising psychiatric diagnoses. The American Journal of Psychiatry, 161(6), 946–54.
Gottesman, I. I., and Gould, T. D. (2003). The endophenotype concept in psychiatry: etymology and strategic intentions. American Journal of Psychiatry, 160(4), 636–45.
Jablensky, A., and Kendell, R. (2002). Criteria for Assessing a Classification in Psychiatry. In M. Maj et al. (eds), Psychiatric Diagnosis and Classification. Chichester, UK: John Wiley & Sons, Ltd, 1–24.
Kendell, R. E. (1989). Clinical validity. Psychological Medicine, 19(1), 44–55.
Kendell, R., and Jablensky, A. (2003). Distinguishing between the validity and utility of psychiatric diagnoses. American Journal of Psychiatry, 160(1), 4–12.
Kendler, K. S. (1980). The nosologic validity of paranoia (simple delusional disorder). A review. Archives of General Psychiatry, 37(6), 699–706.
Kendler, K. S. (2009). An historical framework for psychiatric nosology. Psychological Medicine, 39(12), 1935–41.
Kupfer, D. J., and Regier, D. A. (2011). Neuroscience, clinical evidence, and the future of psychiatric classification in DSM-5. The American Journal of Psychiatry, 168(7), 672–4.
Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports, 3(9), 635–94.
Löwe, B. et al. (2008). Validity of current somatoform disorder diagnoses: perspectives for classification in DSM-V and ICD-11. Psychopathology, 41(1), 4–9.
Mackie, J. L. (1965). Causes and Conditions. American Philosophical Quarterly, 2(4), 245–64.
Messick, S. (1990). Validity of test interpretation and use. New Jersey: Educational Testing Service, Princeton University.
Mill, J. S. (2002). A system of logic. Honolulu: University Press of the Pacific.
Nelson-Gray, R. O. (1991). DSM-IV: Empirical Guidelines from Psychometrics. Journal of Abnormal Psychology, 100(3), 308–15.
Pies, R. (2011). Toward a Concept of Instrumental Validity: Implications for Psychiatric Diagnosis. Dialogues in Philosophy, Mental and Neuro Sciences, 4(1), 18–19.
Pouncey, C. (2003). Diagnosis, classification, and validity: Reliability and sensitivity as test cases. WPA Section on Classification, Diagnostic Assessment and Nomenclature—Newsletter, December 9–11.
Robins, E., and Guze, S. B. (1970). Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia. American Journal of Psychiatry, 126(7), 983–7.
Rodrigues, A. C. T., and Banzato, C. E. M. (2009). A logical-pragmatic perspective on validity. Dialogues in Philosophy, Mental and Neuro Sciences, 2(2), 40–4.
Rounsaville, B. et al. (2002). Basic nomenclature issues for DSM-V. In D. J. Kupfer, M. First, and D. Regier (eds), A research agenda for DSM-V. Washington: American Psychiatric Association Press, 1–30.
Schaffner, K. F. (2002). Clinical and etiological psychiatric diagnoses: Do causes count? In J. Z. Sadler (ed.), Descriptions and Prescriptions. Johns Hopkins University Press, 271–90.
Skinner, H. A. (1981). Toward the integration of classification theory and methods. Journal of Abnormal Psychology, 90(1), 68–87.
Spitzer, R. L. (2001). Values and Assumptions in the Development of DSM-III and DSM-III-R: a Belated Response to Sadler, Hulgus, and Agich. Journal of Nervous and Mental Disease, 189(6), 351–9.
Spitzer, R. L., and Williams, J. B. W. (1985). Classification in Psychiatry. In Kaplan & Sadock’s Comprehensive Textbook of Psychiatry (4th Ed), 591–612.
Stoyanov, D. (2009). The Cross-validation in the Dialogue of Mental and Neuroscience. Dialogues in Philosophy, Mental and Neuro Sciences, 2(1), 24–8.
Szasz, T. S. (1960). The myth of mental illness. The American Psychologist, 15(7), 733–56.
Trafimow, D. (2010). The Implications of Meaning for the Validity of Diagnostic Categories. Dialogues in Philosophy, Mental and Neuro Sciences, 3(1), 23–4.
Zachar, P. (2010). Defending the Validity of Pragmatism in the Classification of Emotion. Emotion Review, 2(2), 113–16.
Zachar, P., and Kendler, K. (2010). Philosophical Issues in the Classification of Psychopathology. In T. Millon, R. F. Kruege, and E. Simonsen (eds), Contemporary directions in psychopathology: scientific foundations of the DSM-V and ICD-11. New York: The Guilford Press, 127–48.
Zachar, P., and Kendler, K. S. (2007). Psychiatric disorders: A conceptual taxonomy. American Journal of Psychiatry, 164(4), 557–65.