United in diversity: Are there convergent models of psychiatric validity?
Drozdstoj St. Stoyanov and Massimiliano Aragona
One of the most significant contributions of the twentieth-century philosopher of science Azarya Polikarov (1974) was his Divergent–Convergent Method (DCM). Published in the Boston Studies in the Philosophy of Science, it was presented as a “heuristic approach to problem-solving” (p. 211). According to Polikarov, scientific problems can be classified into problems of (i) existence, (ii) explanation, and (iii) elimination of contradiction (Entscheidbarkeit). In his scheme, classification and validity belong to the problem of explanation, which includes under its auspices the activities of construction and substantiation.
According to Polikarov, scientific problems are explored in two stages. In the first stage, an initial field of possible or design-solutions is formulated, some of them only hypothetical, others better supported with available evidence. Very often those initial design-solutions are contradictory and mutually exclusive. This divergent stage is developed further as new variant-solutions interact. In the case of psychiatric nosology the initial field is composed of such classical approaches as the categorical model adopted from medicine and the dimensional model adopted from psychology. The new variant solutions which have emerged over the past decades include the prototype, cluster, structural, RDoC, and narrative/person-centered models.
Additionally, Polikarov describes three variations of design-solutions in the divergent stage: radical, moderate, and combined. By radical he means solutions which take an opposite and incommensurable stand with respect to prevailing models. By moderate he means solutions with an intermediate position between the radical and combined. Most modern scientific problems are usually penetrated at the level of combined design-solutions. The latter are subdivided into three kinds:
(i) alternative—without constraint, including radically alternative;
(ii) with weak constraint of diversity, i.e., produced by removal of some variants;
(iii) with strong constraint of diversity, which can entail formulation of unique combinations of solutions.
In Polikarov’s second stage the field is “reduced” (or converged) to a more restricted area of predominant group of project-solutions. This convergence is performed via logical and content considerations, emerging empirical data, and methodological principles. Those principles include: representativeness, simplicity, originality, explanatory and predictive power, coherence, and congruity. The reduction of the field of possible design-solutions eventually results in a dichotomous situation:
(i)there either exists one true solution and all others are false; or
(ii)the true decision is the combined (synthetic) one.
In the case of psychiatric validity, two contrasting possible design-solutions are the realist model and the pluralist–perspectival model. In the realist model, the concepts of validity are unified under one single biologically sound concept and others eliminated as irrelevant. In the pluralist model the concepts of validity can be seen as partially true and context dependent.
Historically, diagnostic systems have pursued a realist concept of validity, i.e., to validate a mental disorder is to show that it is a real entity in nature, and this is best done when we are able to validate it with external criteria such as neurobiological dysfunctions. Those proponents of biological psychiatry that had adopted a reductive approach to the mind–brain relationship expected that technological progress in neuroscience would have offered ultimate explanations and therefore be robust validators for mental disorders. This expectation unfortunately has not been fulfilled. Despite the many encouraging advances in neuroscience, neurobiological markers are still far from being discovered for mental disorders.
The result has led to a crisis of confidence regarding the validity of psychiatric classification. With this crisis in mind, the fundamental queries raised in conceptualizing our monograph were:
1. Is a “Big V” (single, unified) form of validity the only option for psychiatry?
2. Are there alternative models of validity?
3. Does it make sense to talk about validity at all?
As is evident in the preceding chapters, an exciting field of divergent solutions to the problem of psychiatric validity has emerged.
According to Loughlin and Miles, questions about the scientific validity of psychiatric diagnosis derive their meaning from specific conceptions of science, value, and reality. In their chapter they contest the dilemma created by the anti-psychiatry movement in which mental disorders are either real diseases or moral entities based on subjective values. They claim that such a rigid dichotomy is the result of a “scientistic” ideal of objectivity, arguing that the assertion that mental disorders are not real diseases does not lead to moral subjectivism if the value-ladenness of psychiatric diagnosis is based upon value-realism. According to them, validity is not a matter of objective scientific evidence, but of ethical judgment about what is (and should be) the human good.
Other authors focus on validity within science while also examining foundational distinctions. For example, making a strong ontological claim, Sabbarton-Leary et al. assert that not all syndromes listed in the DSM-5 are natural kinds. Only those entities that have a biological cause deserve to be considered “mental disorders,” and only to them does the traditional debate on validity apply. But there are also “mental harms,” and they too are of interest to psychiatry. Mental harms are defined ontologically as “para-natural kinds,” which are states harmful to the agent, without a clear and distinct biological etiology. In such cases validity cannot be a matter of discovering an underlying biological etiology, and other factors such as intensity, duration, or distress may help to determine their clinical relevance.
Thus, while Loughlin and Miles tend to translate the debate about validity into the field of ethics, Sabbarton-Leary et al. accept the traditional realist account for their “mental disorders” strictu sensu, but deny its application to their “mental harms.”
Marková and Berrios examine a more specific question: whether neuroimaging techniques are adequate tools for validating mental disorders. According to their analysis, there is a basic ontological question which is prior to any consideration of validity, namely, what kind of objects are those mental symptoms that we should investigate by means of neuroimaging tools? They argue that mental symptoms are hybrid objects with a physical kernel (the neurobiological signal) enveloped by semantic wrappers (personal, familial, and sociocultural meanings that shape the original experience). Those symptoms where the neurobiological part is prevalent are more apt for neuroimaging exploration, while in those cases in which the semantic construction is more important, imaging will be an inadequate validator.
One question that arises from these reflections is: if validity is a matter of finding a correlation between the index mental phenomena and the neurobiological substratum, is current neuroscience able to validate disorders? This is not an easy question to address. The limitations that serve as barriers to answering this question include limitations in the techniques (e.g. insufficient spatial and/or temporal resolution), in the phenomena to be explored (the discussed ontological differences between mental disorders and between mental symptoms), in the proxy variables reporting such phenomena (e.g., is a score in a rating scale a good substitute for the index phenomenon?), and also in the timing in the assessment procedures.
Stoyanov et al. directly address the last two limitations. They stress that the outputs of rating scales and standardized interviews represent decontextualized excerpts drawn from patients’ narratives, which can hardly correlate with neuroscientific examinations that are performed at different times and within a different domain of knowledge. They show that the findings from psychiatric tools tend to be validated within their own disciplinary field and the findings from neurobiological tools tend to be validated within their own disciplinary field. Stoyanov and colleagues propose to manage such an explanatory gap by implementing a program of cross-disciplinary validation. As an example, mental phenomena should be cross-validated through measurement of the brain activation detected by fMRI and simultaneous administration of state-dependent clinical measures.
The relationship between validity and measurement practices in psychological assessment and psychiatric diagnosis is insightfully explored by Keeley. Validity in psychiatry reflects both the validity of the diagnostic construct (the category, or the dimensional profile), and the validity of the diagnostic process (i.e., the procedure we use to arrive at that diagnostic construct). Keeley argues that the assumptions underlying the measurement process have a role in shaping the structure of the resulting diagnostic entity. In other words, the way measures are constructed, as well as the way they are practically used, both influence the resulting diagnostic entity. For example, if a clinician believes that psychiatric objects are categorical, he/she will adopt assessment instruments (such as yes–no checklists) which are more likely to provide categorical diagnoses. In contrast, a clinician who believes that psychiatric symptoms vary continuously in the population without a clear distinction between normality and pathology will probably use inventories that provide dimensional scores. One important conclusion that can be drawn is that there is no single way to estimate the validity of psychiatric diagnosis. Diagnostic validity is pluralistic and impossible to disentangle from the measurements and pragmatic context in which it is used.
Similar pragmatic considerations are expressed by Aragona, who conceives mental disorders as more or less useful concepts for practical needs, constructed in specific places and times to meet practical needs, and in need of recalibration depending on socio-cultural circumstances and scientific priorities. In this context validation is no longer absolute but relative to the diagnostic system(s) in which validity questions make sense.
In their chapter, Rodrigues and Banzato focus on the pragmatic concept of utility. They distinguish between diagnostic validity and nosologic validity. Diagnostic validity refers to how diagnostic criteria lead to the proper identification of clinical instances of a psychiatric construct. Here they place the notions of face, content, criterion-oriented, and construct validity. Nosologic validity refers to the justification for including a diagnostic category in a system of classification. They distinguish between two different conceptions of nosologic validity. According to the realist conception a diagnostic category is valid if it represents a real entity. According to the pragmatic conception a diagnostic category is valid if it is useful. They stress that reality and utility cannot be reduced to one or the other, and thus a “dual-track” program of validation is needed.
Pragmatic accounts of validity are further addressed by Murphy, who critically discusses Schaffner’s pragmatism. Schaffner opposes making a firm distinction between realist validity and utility on the grounds that utility is constitutive of reality (thus, a position which is opposite to the “dual-tack” validation proposed by Rodrigues and Banzato). Against Schaffner, Murphy argues that a disorder’s dependence upon being recognized under some concept does not mean that what is perceived via concepts does not exist. Despite this conceptual influence, he argues that we have a “relatively direct access to the world” which is sufficient for a realist account, provided that the realist accepts auxiliary hypotheses and can discard plausible alternatives.
Murphy’s realism holds that psychiatric constructs are related to the causal structure of the world. Drawing on Cummins’ concept of dysfunction, which he calls “the systemic view,” Murphy argues that although mental disorders can be the effects of biological dysfunctions, the concept of dysfunction itself is not a mere matter of fact. Rather, dysfunctions are intrinsically normative concepts. We judge something to be dysfunctional because it does not conform to our ideas about how a good-functioning system should be. Like weeds, mental disorders are something that we don’t like because of certain interests we have. We know scientific facts about them, but there is nothing intrinsically disordered independently from our negative evaluations. Normative judgment comes first, and normative issues being open, we cannot validate a diagnosis; we can just correlate it with part of the world’s structure.
Other interesting remarks about the intrinsic limitations of validity intended as the discovering of the neurobiological etiology are offered by James Phillips. After distinguishing the validity of the diagnostic constructs from the validity of the diagnostic criteria and the validity of the external measures used to confirm a diagnosis, Phillips considers three different kinds of diagnostic validity: “Strong Syndromal Validity,” “Weak Syndromal Validity,” and the validity of the RDoC project. According to Phillips, all three kinds of validity have the same limitation: they depict psychiatric disorders as clocks whose parts can be studied independently and mechanically disassembled. The only difference would be that the Strong and the Weak Syndromal Validities focus on syndromes, while the RDoC project focuses on symptoms.
Against this common mechanistic view, Phillips argues that mental disorders are complex systems, meaning that the actual function of their parts depends upon the total organization of the system. Complex, interactive etiology can produce heterogeneous presentations and high levels of comorbidity, which influences our efforts to classify mental disorders. Additionally, Phillips introduces another important theme, i.e., the role of psychological factors. While the debate on validity is usually shaped in neurobiological terms, Phillips argues that a complete biomedical model would take the complexity of psychiatric disorders into account, and in doing so have to include psychological domains among its validators.
The theme of psychological and psychosocial factors leads us to a related issue, that of “personal” features. In their chapter, Mezzich and Salloum propose a person-centered integrative diagnosis as an alternative to the reductionism which is implicit in the concept of diagnostic validity. They argue that good diagnosis must go beyond the nosological diagnosis (i.e., identifying a diagnosis as a general entity pertaining to several patients) to also involve understanding the person who is presenting for care. They propose an integrative model in which the nosographical level (the categorical attribution of a formal ICD diagnosis) is only one part of a larger assessment procedure including whole health (both ill and positive health), dimensional profiles, and narrative accounts of the patient’s values and of cultural experiences of both illness and health.
That personal and transnosographic factors have to be considered, beyond the categorical diagnosis, is exemplified by two chapters that utilize personality as exemplar mental disorders. Jacobs and Krueger contrast the current top-down approach to psychiatric diagnosis to what they call “structural validity.” The top-down approach is criticized because it starts from expert opinions instead of starting from scientific evidence. According to them, the top-down approach, focusing on reliability and external validity, has led to artificial distinctions between disorders that are actually empirically linked. This has resulted in mixed cases, overlapping boundaries, and high comorbidity. Moreover, they argued that external validity is too broad and unspecific to be the sole source of validation of a diagnostic system. The structural validity alternative is a bottom-up data-driven procedure. In this approach the categorization of mental disorders would reflect the way that disorders are organized in nature. For example, the DSM-IV personality disorders have been discredited because their comorbidity is excessively high, thus undermining the categorical assumption that they are distinct disorders. In a structural validity approach the focus would be on a “trait-based” system because such a system is better able to comprehensively model the symptom space and to correspond to the patterns that exist in patient populations.
Another personality-oriented alternative is presented by Cloninger, who criticizes both the reification of current psychiatric categories (considered by many as discrete disease-entities) and those approaches that start from personality traits considered as separate units that can be added together. In his psychobiological approach, which shows similarities with the Person-centered Integrative Diagnosis described by Mezzich and Salloum, Cloninger proposes to integrate category-like prototypes, dimensions, configural profiles, and person-centered narrative accounts. In doing so, he considers the person’s self-organizing goals and values and their role in health promotion, as well as in the vulnerability to personality disorders and psychopathology. In this context, validation is not a purely logical or empirical process. Rather, it requires an integration of intuition, reasoning, and observations to test, correct, and refine the initial insights. From this perspective, validation is an open process building on insights coming from different disciplines (biological, psychological, social, cultural, philosophical, and spiritual).
Finally, Muller criticizes the operationalized, neo-Kraepelinian DSMs for being not valid because they ignore what is essential in psychiatric diagnosis: namely, that a valid paradigm should be based on a valid anthropology, acknowledging the role of freedom, responsibility, and self-deception in the genesis of mental illness. Having renounced the search for the essence of mental phenomena in favor of an agreement on superficial descriptions, the DSM, he asserts, has failed. Muller’s alternative model, called the “Four Domains of Mental Illness” (FDMI), is based on recent perspectivist theories of psychiatric illness and is deeply influenced by Meyer’s psychobiology, existentialism, and pragmatism. In this model truth is made in the course of experience, and the validity of a psychiatric classification largely depends on its ability to provide an understanding of the person’s experience and behavior that both makes sense and helps psychiatrists achieve their professional goals.
The extensive field of the divergent perspectives on psychiatric validity is produced by interactions with novel variant solutions (e.g., structural validation versus person-centered diagnosis). Divergent design solutions without any constraint of diversity have practically led to the status quo of psychiatric validation. The clearly understood concept of reliability has been emphasized instead of validity, in part, because of the incommensurability of the alternative validity concepts. Polikarov’s strong constraint of diversity in design-solutions and eventually one single realist model are still far out of reach and most probably impossible. Therefore, the most likely scenario would be to limit psychiatric validity to a weak constraint of diversity in the divergent stage of problem solving.
Further, a reduced field of predominant project-solutions to the problem of validity in mental health should be pluralistic, therefore a combined (synthetic) solution in terms of the divergent convergent approach. This combined project should be based on the following considerations, drawing from the following agreements shared by two or more contributors:
1.Current classification systems such as the DSM and ICD seem to be unsuccessful attempts to establish validity by convention (Aragona, Cloninger, Jacobs, and Krueger). Current classification systems were unsuccessful because they define mental disorders on the basis of superficial symptoms without considering their role in the person’s mental structure and their meaning for the person. This led to heterogeneous categories that are not easy to correlate to external validators, with too many interfering variables contemporarily at work.
2.The categorical and semi-dimensional (criterion) approaches to psychiatric nosology have neither scientific validity nor clinical utility since they miss important facets of knowledge about mental disorder (Sabbarton-Leary et al.; Rodriguez and Banzato; Phillips).
3.The person of the patient-in-context should remain the focus of clinical interest, instead of any form of presumed formal criteria for diagnosis (Cloninger et al.; Mezzich and Salloum).
4.Broader, “high umbrella” approaches to validation should be employed in the first place, such as prototype, dimensional, and person-centered (Mezzich and Salloum; Jacobs and Krueger; Cloninger).
5.Diagnostic entities should be underpinned with sound evidence from neuroscience. However, the current status of the connections between neuroscience and psychiatry is insufficient due to essential methodological gaps (Markova and Berrios; Stoyanov et al.). Therefore, the framework of joint neuroscience–psychiatry scientific inquiry should be revisited to provide stronger inter-disciplinary research findings.
In conclusion, validity is a manifold concept. The different options presented in this book can be arranged along a dimensional logical space having two extremes. One is a radical pluralistic approach where every perspective stipulates a unique concept of validity that is to some extent incommensurable with other concepts of validity. In this relativistic case, there is no common ground to decide between alternative validation procedures. Any concept of validity would be fine in its own domain, but any domain could learn nothing from the debates taking place in other disciplinary fields.
The other extreme is a radical unifying approach aiming at finding the only right concept of validity, which should be used by everyone. Although this approach does not need to be a realist one (the unique acceptable definition might also be ethical, or logical, etc.), it is always reductionist, i.e., alternative concepts have to converge on the correct concept. There are three problems with this approach. The first problem is about the preferred level of analysis: should we base the concept of validity on neurobiology and at what level of neurobiology? The second problem is about feasibility: are we ready to decide what concept of validity is the right one? The third problem is desirability: are we sure that we would not lose important phenomena by converging all levels into one preferred level?
Several possibilities occupy the middle ground between these extremes. These include what we might call “moderate perspectivism” and “moderate convergentism.” According to moderate perspectivism, every domain of knowledge has its own methodologies and core assumptions, and thus its own concept of validity, but nevertheless concepts can be contrasted across domains. As a result, different disciplines can interact and reciprocally influence each other through dialogue. According to moderate convergentism, although a unique validity concept is probably unattainable, nevertheless the plethora of validity proposals might be reduced, focusing on shared similarities and cross-validation procedures.
One contribution of this book is that it provides resources for (i) those seeking a unified realistic model, as well as (ii) those that want to maintain a pluralistic approach to knowledge domains that they consider separate but interacting. Hopefully we have managed to provide a cognitive framework that may inform future efforts toward the reappraisal of validity and validation standards in the mental health disciplines.
Polikarov, A. (1974). The Divergent–Convergent Method. In R. S. Cohen and M. Wartofsky (eds), Methodological and Historical Essays in the Natural and Social Sciences, Boston Studies in the Philosophy of Science, Vol. XIV, Dordrecht, 213–33.
Stoyanov, D. S. (2010). On the epistemological legacy of A. Polikarov. Epistemologia, XXXIII (2010), 135–8.