Quantitative analysis has many limitations. In this paper, Siegfried Kracauer proposes that qualitative analysis may be a more fruitful procedure in some stages of international communications research.
Siegfried Kracauer, formerly a prominent journal editor in Germany, has published widely in the field of communications research. He is presently completing a book on the aesthetics of the film.
This paper submits three propositions regarding the significance of qualitative exegesis for communications research:
1. One-sided reliance on quantitative content analysis may lead to a neglect of qualitative explorations, thus reducing the accuracy of analysis.
2. The assumptions underlying quantitative analysis tend to preclude a judicious appraisal of the important role which qualitative considerations may play in communications research. Hence the need for theoretical reorientation.
3. The potentialities of communications research can be developed only if, as the result of such a reorientation, the emphasis is shifted from quantitative to qualitative procedures.
Overemphasis on quantification tends to lessen the accuracy of analysis. Content analysis is frequently obliged to isolate and process the more intricate characteristics of a sample; and whenever this happens it runs the risk of treating them inadequately. Quantitative analyses for example, commonly attempt to determine the “direction” of a communication, i.e., the extent to which it is “for,” “against,” or “neutral” in regard to a given subject. In such instances coding is often performed on the basis of a graded scale which defines a continuum ranging from “very favorable” to “very unfavorable,” from “very optimistic” to “very pessimistic,” or the like. Some quantitative analysts admit, however, that despite such scales, direction “is not always easily analyzed in an objective fashion.”1 The actual rating of a given unit of the communication on one or another step of the continuum still involves qualitative considerations which may bear on the whole of the communication. Unless the communication is a peculiarly one-dimensional affair, these assessments require a great deal of circumspection and delicacy. In quest of reliability, the quantitative analyst may—and frequently does—therefore introduce elaborate directives, to permit selected coders to arrive at highly reliable decisions. Such a breakdown of a complex direction continuum into relatively elementary scales inevitably invites simplifications apt to blur the picture. They render arbitrary, for example, the real gap between “very favorable” and “favorable”; and they place under one uniform cover (e.g., “favorable”), a great variety of treatments whose differences are perhaps highly relevant to the purposes of the analysis.
At this point the objection may be raised that it is possible to attain any degree of precise distinction by introducing sufficiently subtle scales in sufficient number. Coders might be trained, for example, to distinguish between matter-of-fact neutralism and well-balanced neutralism. Yet even the most refined tools of measurement may not enable the analyst to reconstruct the direction of the original communications. His rigidly atomistic data are likely to preclude inferences as to the way in which the data are interrelated. Significantly, it is this very interrelationship which often contributes largely, and sometimes definitively, to determine the direction of the overall text. Gestalt psychologist or not, any literary critic knows that, due to their organization, communications often move in a “direction” at variance with what a computing of the directions of their elements would yield. In such cases precise quantification, used alone, will actually encourage inaccurate analysis.
Of course, it is theoretically conceivable that the content analyst might succeed in quantifying the interrelationships between the “plus” and “minus” units of the communication, and so be enabled to measure direction correctly. But such a procedure would necessarily involve categories in such number and of so refined a nature that the incidence of their use would often be minute. Since, with the decrease of sizable frequencies, qualitative appraisals play a larger role in interpretation anyway, there is no reason why such cumbersome quantitative techniques should be preferred to qualitative exegesis proper. At best they would lead, in a very roundabout way, to what the latter could disclose without unnecessary complications. Quantitative analysts in fact recognize the danger of “over-fine” categorization, and continually caution against it. And yet to avoid it is to run the risk of oversimplifying the more intricate characteristics of many communications.
Direction is by no means the only contextual characteristic which resists a breakdown into easily countable components or even the development of “indicators” that permit the unambiguous, let alone exhaustive, identification of such components. Suggested procedures are often inadequate. Berelson, for example, suggests that “sophistication” can be quantitatively analyzed by “the indicator of the amount of qualifications appearing in the content (‘on the other hand,’ ‘however,’ ‘although’).”2 Even granted that the indicator “qualification” is an adequate index of the particular form of sophistication in the given text (and certainly this could not be true for all texts), the number of qualifications still need not indicate the degree of sophistication, which might rather depend, for example, on the intrinsic nature of the qualifications themselves. The analyst might, of course, break down their “nature” itself into quantifiable elements, but such a procedure would lead straight to the dangerous complications already discussed.
Since most communications include intricate characteristics, and since many of the hypotheses which prompt analysis cannot help drawing on them, it would appear that many quantitative investigations include frequency counts which rest on uncertain ground. Yet once the figures are secured from the material, they are as a rule taken for granted; in fact, the analysis often uses them as a base for statistical elaborations. Probabilities are calculated; correlations are established and interpreted. Since these operations evolve on a mathematical plane—that is, without further recourse to the content analyzed—it is possible that their results are more inaccurate and oblique and less truly representative of the communication than are the doubtful counts from which they take root.
Among the assumptions underlying quantitative analysis two are of special interest in that they tend to preclude a judicious appraisal of the role which qualitative consideration might play.
There is first the basic assumption that, due to its quantifications and counts, quantitative analysis is the only possible objective systematic and reliable analysis of content. Many researchers consider this as axiomatic.3
The second assumption which is relevant restricts the meaningful application of quantitative techniques to communications whose manifest content does not lend itself to being interpreted in different ways. Berelson, for example, proposes:
“If one imagines a continuum along which various communications are placed depending upon the degree to which different members of the intended audience get the same understandings from them, one might place a simple news story on a train wreck at one end (since it is likely that every reader will get the same meanings from the content) and an obscure modern poem at the other (since it is likely that no two readers will get identical meanings from the content).… The analysis of manifest content is applicable to materials at the end of the continuum where understanding is simple and direct, and not at the other. Presumably, there is a point on the continuum beyond which the ‘latency’ of the content (i.e., the diversity of its understanding in the relevant audience) is too great for reliable analysis.”4
These assumptions put communications research, particularly applied communications research, in an awkward position. While it may be able to avoid obscure poems, it is much concerned with texts in which latent meanings not only pervade the manifest content, but also are intricately related to the objectives for which the analysis is undertaken. Such latent elements may strongly resist quantification, and occasionally the quantification is actually foregone. For example, the Bureau of Applied Social Research at Columbia University bases its recent studies of communications habits along the Soviet periphery on interviews which involve the respondents’ total life to such an extent that practically no word in the interview record is free of multiple connotations. Accordingly, the studies do not confine themselves to quantitative measurements but also analyze, in purely qualitative terms, the intrinsic essence of certain interviews, the possible significance of deviant attitudes, etc. That these qualitative explorations are often touched off by statistical accounts should not blind one to the cases in which they expose unique characteristics without regard for frequencies and the like. All in all, the nonquantitative part of the studies enjoys relative independence, in keeping with the character of both the interviews and the various hypotheses bearing on them. Analysts of international communications have likewise often found themselves in need of qualitative procedures. When an area specialist, for example, is asked to estimate the presumable effectiveness of certain themes and of the devices employed to get them across, he is forced to focus on characteristics and interrelations which it would be meaningless to count because of their highly individual nature.
It is inevitable that the champions of quantitative analysis should regard such nonquantitative explorations as precarious adventures in uncontrollable intuition rather than procedures of verifiable research. The common objection to these procedures is that they are “impressionistic,” “unobjective,” and lacking in “verifiable evidence.” Such criticism follows logically from the basic assumptions of quantitative analysis. But what about the assumptions themselves?
There is little doubt that quantitative analysis is meaningful if it keeps to communications at the extreme end of the continuum defined by descriptions of train wrecks and similar events. There intricate characteristics hardly enter the picture; should such a category as “direction” be needed at all, it would have to cover only the most elementary pros and cons. And their frequency counts are of major importance. Within this border region quantitative analysis is indeed the only objective, systematic, and reliable procedure of analysis. (It is not, however, necessarily exhaustive. If, for example, the content is set in historical perspective, its “latency” will immediately increase so that quantitative procedures no longer suffice to describe it adequately.)
Yet quantitative analysis does not confine itself to inquiring into these extreme cases. It is often applied to content somewhere along the continuum, content which, though not as obscure as the modern poem, is nevertheless more involved and allusive than the reports at the opposite pole. In addition, such investigations may seek to trace attitudes and interests of population groups, to determine the psychological states of persons and groups, to discover stylistic features—purposes which force the investigator to examine characteristics rarely found in the train-wreck region. But if quantitative analysis expands beyond the confines set up by one of its underlying assumptions—thus running all the risks discussed in the first section of this article—then the other basic assumption that it is the only objective and reliable analysis of content cannot be upheld either. As currently practiced, quantitative analysis is more “impressionistic” than its champions are inclined to admit. All of them, incidentally, readily grant the need for qualitative reasoning in the initial stages of category formation.5 They more rarely admit, however, that the quantification processes themselves often require much conjecturing which is not in actuality tied to objective, impersonal definitions.
A recent “quantitative” analysis of Voice of America and BBC broadcasts classifies the “style” of contextual units as “matter-of-fact,” “mildly emotional,” and “highly emotional.” Granted that this particular classification was labelled as experimental and the data as merely suggestive, it is nevertheless significant that the quantitative analysts made the classification “mainly with reference to value-laden terms,”6 the emotional intensity of which they attempted to assess. Certainly no procedure could be more impressionistic. In addition, quantification by this particular indicator promotes a peculiarly fragmentary view of style. For certainly the very absence of value-laden terms in, for example, a sober announcement of the fall of a city, an army, or an individual may constitute a “matter-of-factness” which is in effect highly emotional.
The example is not unique. Numerous quantitative analyses are similarly threaded with impressionistic judgments. And these judgments may in fact be more unaccountable than those found in communications studies of a predominantly qualitative nature. For within the framework of quantitative analysis, qualitative exegesis is condemned to playing a black sheep role. Recognized mainly as a means to arrive at suitable quantifications, its use in analysis proper is regarded as shameful, and may in fact be pursued with guilty haste and lack of discipline. This explains why the qualitative considerations on which most content analysis studies must draw for classification so often fail to penetrate the given text. Being no end in themselves, they threaten to turn into opinion-laden short cuts. The reproach of impressionism which determined quantitative analysts direct against nonquantitative insight thus tends to boomerang. Quantitative analysis is in effect not as objective and reliable as they believe it to be.
Since quantitative analysis proves to be inadequate to describe more involved communications, it would seem advisable to inquire into the prospects of an analytical approach which emphasizes qualitative rather than quantitative procedures. Can we assume that such an approach is more adequately descriptive? And, if so, what of its scientific relevance? Will its “impressionism,” its “inevitable lack of objectivity,” nullify the advantages it may otherwise offer?
Before considering these questions, it should be emphasized that the terms “qualitative analysis” and “quantitative analysis” do not refer to radically different approaches. Quantitative analysis includes qualitative aspects, for it both originates and culminates in qualitative considerations. On the other hand, qualitative analysis proper often requires quantification in the interest of exhaustive treatment. Far from being strict alternatives the two approaches actually overlap, and have in fact complemented and interpenetrated each other in several investigations.7
Qualitative analysis by definition differs from quantitative analysis in that it achieves its breakdowns without special regard for frequencies. What counts alone in qualitative analysis—if the verb is permissible in a context which defies counting—is the selection and rational organization of such categories as condense the substantive meanings of the given text, with a view to testing pertinent assumptions and hypotheses. These categories may or may not invite frequency counts. In order to demonstrate the greater adequacy of qualitative analysis to communications which exceed straight reporting, these two possibilities will be dealt with separately.
In the case of categories which do invite frequency counts, there is no real difference between the qualitative and quantitative approaches. In theory, both might produce identical classifications. And yet the odds are that the qualitative analyst will be the less inhibited of the two in discovering countable characteristics. As Berelson points out, qualitative studies usually focus not so much on the content of a communication as rather on its underlying intentions or its presumable effects on the audience; “quantitative analysis” on the other hand, “is more likely to focus first upon the straight description of the content itself, if for no other reason because of the amount of energy devoted to the counting procedure.”8 The more involved communications, however, reverberate with so many latent meanings that to isolate their manifest content and describe it in a “straight” manner is not only almost impossible, but can hardly be expected to yield significant results. Such a focus on manifest content everywhere implies a naive extension of the limits implicit in the assumption, per se legitimate, that quantitative techniques are meaningful at the train-wreck end of the continuum. This explains why the qualitative analyst is in a better position than the quantifier to trace relevant characteristics which admit of frequency counts. Free of any biasing prepossession with manifest content, the qualitative analyst explores the whole of the content in quest of important categories. And since he devotes all his energies to this quest, he stands a good chance of coming inadvertently across frequency categories which might have eluded his grasp had he been preoccupied with quantifications at the outset. People often find in passing the very things they have sought in vain.
Examples bearing out these observations are extremely rare because practically no texts have been subjected to independent analyses of both the quantitative and qualitative type. It is perhaps relevant, however, to note that although quantitative analyses have occasionally attempted to employ categories dealing, at least experimentally, with presumed effects, such categories have dealt almost exclusively with manifest aspects of atomistic units of the texts. The previously cited category of “style” in international communications is a case in point. Qualitative analyses of similar material have also framed quantifiable categories dealing with the “structure” of the text as a whole, i.e., the linkage, manifest or latent, which makes the atomistic units a Gestalt. Such freedom to seek and use quantifiable categories of latent content has, at least to date, been almost exclusively characteristic of qualitative exegesis.
Thus, qualitative analysis steals a march over quantitative analysis in fields common to both—i.e., in regard to categories which do invite frequency counts. But by virtue of its ability to use non-quantifiable frequencies, qualitative exegesis also penetrates textual dimensions which are completely inaccessible to quantitative techniques. An example of the limitations placed on quantitative analysis may be found in Berelson’s statement that “Whenever one word or one phrase is as ‘important’ as the rest of the content taken together, quantitative analysis would not apply.”9 Qualitative exegesis would; and it would make its breakdowns hinge on this one word or one phrase. As a case in point, let us suppose that an international communicator wished to ascertain whether his texts evidenced respect for the audience. A good indicator of this characteristic, though certainly not the only one, is the way in which the communicator refers to his listeners. It is immediately evident, however, that neither the relative number of laudatory and critical references, nor distinctions between “moderate” and “excessive” praise or blame will give any valid picture of the degree of esteem in which the audience is actually held. Frequency counts will reveal the amount of different modes of praise or blame, but since any mode may spring from various psychological sources, the counts are unlikely to yield information about the characteristic “respect” itself. The absence or presence of respect could obviously be better inferred from the manner in which the positive and/or negative references to the audience are interwoven; recognizable patterns of reference would no doubt appear in the communication. Qualitative exegesis would attempt to bare these patterns and assess their presumable significance for the characteristic under consideration. This particular task would be facilitated by the common awareness that certain familiar patterns of interwoven praise and/or censure—”ideal types” in Max Weber’s sense—are symptomatic of respectful or disrespectful conduct. For instance, a balanced mixture of friendly approval and frank censure, both being voiced on fitting occasions, would clearly indicate that the communicator is treating his audience as he would a friend or peer; conversely, a pattern of abrupt alternation between extreme praise and harsh criticisms or threats would indicate that the communicator was bluntly trying to manipulate the minds of his audience, which in turn would indicate his low opinion of their independence and dignity. It is particularly to be noted that one single instance of such a configuration of statements would suffice to color the entire communication. In reference to such characteristics, frequency counts are of little relevance. What is relevant are the patterns, the wholes, which can be made manifest by qualitative exegesis and which can throw light upon a textual characteristic which is allergic to quantitative breakdowns.
Unlike quantitative techniques, which draw guiltily upon hasty and incomplete impressionistic judgments, qualitative analysis is frankly and resolutely impressionistic. And it is precisely because of its resolute impressionism, that qualitative analysis may attain to an accuracy which quantitative techniques, with their undercurrent of impressionistic short cuts, cannot hope to achieve. Carrying its explorations beyond the point at which many content analysis investigations prematurely stop, as if fearful of drifting too far from the secure haven of statistics, qualitative exegesis is indeed capable of classifications and descriptions which conform far more closely to the texts than those commonly produced by quantitative analysis. The relative capabilities and limits of these two approaches are nowhere better manifested than in the frequent failure of full-blown quantitative studies to achieve the brilliant promise of their pilot or exploratory stages. The pioneering steps, performed on a small sample, invite attention to unique traits which are perhaps manifest in only one single configuration of statements. The insight into wholes which these unique patterns provide gives rise to observations and hypotheses of unusually rich relevance. The pilot study is, in fact, a model of qualitative exactitude and circumspection. But in the fuller study which follows, the development and testing of these rich hypotheses is entrusted to systematic quantification, in which both infrequencies are deemphasized, and the original overtly impressionistic and accurate insights are not developed for lack of the very spirit in which they were conceived.
One might ask, of course, whether the superior precision attained by qualitative procedures is not bought at too high a price. For it is true that qualitative analysis, being inevitably subjective, cannot ascertain the accuracy and validity of its findings in the manner of an exact science. One and the same topic may invite different qualitative appraisals of almost equal plausibility; and no accumulation of evidence will determine, in an objective way, which is closer to truth. But though there is no objective truth in this field, the lack of it does not entail lawlessness; qualitative analysis is not a discipline that admits arbitrary speculations. The believers in exact science among the social scientists are inclined to exaggerate, along with the objectivity of quantitative analysis, the dangers which qualitative techniques incur because of their subjectivity. Any historical period produces only a limited number of major philosophical doctrines, moral trends and aesthetic preferences, and if qualitative analysis operates, as it should, below the level of sheer opinion, these influences can be discerned and controlled. Moreover, communications which are sufficiently outspoken to canalize the imagination usually prove a powerful factor in bringing about a convergence of viewpoints and approaches. It is therefore a reasonable guess that different analysts will arrive at similar conclusions with regard to many texts. An experiment to test the guess is now being designed.
Finally, one may legitimately ask whether communications research, as such, should really try to match exact science. Documents which are not simply agglomerations of facts participate in the process of living, and every word in them vibrates with the intentions in which they originate and simultaneously foreshadows the indefinite effects they may produce. Their content is no longer their content if it is detached from the texture of intimations and implications to which it belongs and taken literally; it exists only with and within this texture—a still fragmentary manifestation of life, which depends upon response to evolve its properties. Most communications are not so much fixed entities as ambivalent challenges. They challenge the reader or the analyst to absorb them and react to them. Only in approaching these wholes with his own whole being will the analyst be able both to discover and determine their meaning—or one of their meanings—and thus help them to fulfill themselves. Far from being an obstacle, subjectivity is in effect indispensable for the analysis of materials which vanish before our eyes when subjected to a treatment confounding them with dead matter. Quantitative analysis is not free of such nihilistic influence. Many quantitative investigations in effect mark the spot where a misplaced desire for objectivity has failed to reveal the inner dynamics of an atomized content.
One final suggestion: a codification of the main techniques used in qualitative analysis would be desirable.
NOTES
1. Bernard Berelson, Content Analysis in Communications Research (Glencoe, IL: Free Press, 1952), 151.
2. Berelson, Content Analysis in Communications Research, 163.
3. Berelson states: “By definition, content analysis must be objective” (171).
4. Berelson, 9–20.
5. Thus, Berelson and Lazarsfeld suggest that the analyst try to formalize into categories his “general subjective impressions” of the content, and that he then put the formulations aside and later come back to them afresh. Bernard Berelson and Paul F. Lazarsfeld, The Analysis of Communication Content (New York: Bureau of Applied Social Research, 1948), 115–17.
6. Marie Jahoda and Joseph T. Klapper, “From Social Bookkeeping to Social Research,” in this issue of Public Opinion Quarterly, 623–30.
7. E.g., the previously cited studies of communications behavior along the Soviet periphery; also Löwenthal’s study, “Biographies in Popular Magazines” and Arnheim’s “World of the Daytime Serial,” in Paul F. Lazarsfeld and Frank N. Stanton, eds., Radio Research I942–I943 (New York: Duell, Sloan & Pierce, 1944).
8. Berelson, Content Analysis in Communications Research, 122.
9. Berelson, 20.