Save the Data

Numbers press in on us from all sides so as to inform. For instance bottles of orange juice display the daily recommended amount of vitamin B2 – but why “2”? And consumers have no idea as to what happens when we fall below this amount – and whether it is even probable that we might indeed fall below this amount. This is precisely similar to those numbers informing us as to the quality of equipment. Who knows what an “unweighted signal-to-noise ratio” is or at what point your number of megapixels becomes superfluous in sharpening your image definition. Or take cholesterol values. Who among us can say how many there are, how the limit-values (which people are warned against) have been calculated, and whether the current research has a clear position with respect to cholesterol? After all these measurements and recommendations we are often highly informed but none the wiser, perplexed even, and in most instances it would be a fallacy to believe that further information would help remedy the situation.   

But at this point an individual can either sit down and spend an evening with Wikipedia, working their way from “riboflavin” through “hypovitaminosis” to “orange juice,” or they can simply shrug their shoulders and continue to live their lives without benefit of this critical information. Yet cryptic clouds of numbers have been hovering over public life for the past several decades, they enter into political decisions, and govern virtually every organization. Nothing is determined or decided without benefit of indicators, benchmarks, quotas, demoscopic values, ratings and rankings. Schools and school systems are tested to rank their effectiveness; banks and industrial concerns are assigned numbers that measure their creditworthiness; in applying for jobs and fellowships, scholars list their Hirsch Index so as to indicate with what regularity their publications are cited. There are also corruption indexes, freedom indexes, happiness indexes for entire countries – and if the Organization for Economic Cooperation and Development (OECD) calculates that in one of their member-states the number of young adults attending the university is far below average, no one asks what they might be doing instead but simply how this number can be increased.  

It was some time ago that the sociologist Wendy Espeland of Northwestern University together with her colleague Michael Sauder of Iowa carried out research on how ranking tables that rate organizational quality will then impact organizations when no one can precisely say just what the figures imply. Espeland and Sauder employed the example of North American law faculties to show how ranking tables that recorded insignificant differences between these faculties can have considerable consequences. Students, parents, and administrations for instance were fixated on the ranking and not on the “actual” differences. Rankings impact a person’s perception of quality. One hears that a certain law school is among the very best and this rubs off even on those researchers who have done little to conduce to such, or it conditions an evaluation of the school’s graduates even though the quality of research was being measured. But most of all rankings cause those being evaluated to adapt themselves to the perspective of the appraisal even if this means abandoning certain idiosyncratic traits and conforming with the norm. One takes those numbers seriously that one doesn’t actually take seriously. 

One need merely call to mind the debates triggered by Germany’s ranking in the PISA study to see that such mechanisms do also have an impact in this country. Yes, it would indeed seem that evidence for the quantified bases of decisionmaking is urgently required in view of the significant role that rankings also play in educational systems where no large private capital-investment decisions are at stake with respect to what university someone should attend. The logic of one’s place in the standings thus applies to completely different decisionmaking situations. It is not least owing to the mass media’s eagerness to bring a constant stream of news to the public that numbers, because of their malleable character, are popular. 

Wendy Espeland has organized a focus group at the Wissenschaftskolleg that is investigating the significance of quantifying techniques in social life. Among the members of this group is the historian of science Theodore M. Porter, who works at the University of California, Los Angeles, and whose standard works Trust in Numbers and The Rise of Statistical Thinking showed that he was among the first to perceive that the impact of “social numbers” does not solely depend on how reliably they have been ascertained. In the structure of the modern state the notion that social life can be calculated plays as decisive a role as it does in the promises of natural and social scientists that they could supply reliable means for impersonal decisions. Porter is presently investigating the use of statistics in psychiatric asylums, so called “madhouses”, of the nineteenth and twentieth centuries. Because many sufferers were permanently lodged there, there was a high density of patient-observations possible. Evidence-based medicine, of which there is so much talk today, had its origins in the statistical evaluations to which early neurologists submitted their own actions. That was extremely unusual; for the physician and his practice it was the individual patient and not the average patient which was the determining factor. At the same time, and long before genetics – and this is what Porter is investigating at the Wissenschaftskolleg – asylums gathered information on causes provided by patients and their relatives and compiled tables that emphasized the causal role of heredity. The data appertaining to today’s extremely data-driven discipline of genetics did not originally come from the laboratory but rather from statistics.   

One reason for the secular increase of numbers-based decisions is the division of labor in society. Or more simply put: specialization spawns knowledge problems. The more specialists there are, the more often must decisions – for instance regarding the apportionment of funds, legal configurations, and political desiderata – be made by people who have no knowledge of what they’re deciding. Numbers, if only in an illusory manner, help to bridge such ignorance, among other reasons because it doesn’t take a genius to ascertain which number is the larger of the two. Numbers also bridge differences in social context. It is through the use of such indicators as “return on sales” and “proportion of women” that one can compare hospitals, tire factories and museums. Wendy Espeland’s first studies investigated those cost-benefit analyses of public projects that assist politicians in making decisions by presenting hypothetical willingness-to-pay scenarios. Plans such as “Stuttgart 21” are substantiated through just such appraisals, in which a vast amount of data is assembled pertaining to economies of time, environmental assessments, investment confidence, purchasing power, noise pollution, etc., all of it approximations and all of it undertaken with heroic assumptions (e.g. perfect information, “complete markets,” unbiased answers) but then all of it finally issuing in exact numbers. 

At the Wissenschaftskolleg, in a number of case-studies, Espeland has (together with Stuart Michaels at the NORC, Chicago) been examining how the measurement of certain facts itself often spawns these facts. An example of this, from 1948, is Alfred Kinsey’s published assertion (based on more than five thousand interviews) that some ten percent of American males between 16 and 55 practiced more or less exclusively homosexual sex. For some years it was primarily this number that was tossed around by the general public – in part, presumably, because it was easy to remember. The year 1948, with its ten-percent-finding, was the genesis of a homosexual minority-consciousness that developed into a social movement. In many contexts it was no longer asked how many homosexuals actually existed – among voters, soldiers, political-party members it was always ten percent! 

Another of Espeland’s case-studies examines how double-entry bookkeeping contributed to the emergence of capitalism. Max Weber is famous for his thesis that in order for this economic form to emerge you needed a specific mind-set expressive of personal qualities such as thriftiness, a restless commitment to acquisition, and a calculable lifestyle. But perhaps this thesis underestimates the importance of social numbers; perhaps they are a surrogate for character. Bruce G. Carruthers (also a sociologist and likewise from Northwestern University) is of the view that what objectively appears to be a consequence of the division of labor could, from a social perspective, be called a crisis of confidence. Whenever a decision must be taken in vague circumstances, the primary basis of any decisionmaking process – familiarity with the issue at hand – is lacking. Seen from this perspective, numbers are a functional surrogate for trust in individual persons, they formalize reputation. 

Carruthers seeks to understand this through a historical examination of the change that creditworthiness has undergone over the years. Whereas credit standing was once primarily based on an evaluation of a borrower’s “business ethics,” for some time now his reliability has been determined through a test system based on economic indicators and legal prerequisites. This test system, embodied by the rating agencies Moody’s and Standard & Poor’s, engenders so much trust that even its drastic miscalculations on the eve of the world financial crisis have not shaken people’s confidence in it in any long-term way.      

Implicit in every study of quantification is a diagnosis of the times. It was in the late 1970s and 1980s that such indicators emerged. Are we perhaps talking here about a component of what the French sociologist Emmanuel Didier (CNRS, Paris), a member of the focus group who is researching “benchmarks,” assumes would be summed up under the somewhat simplistic heading of “neo-liberal” policy. But at the same time our handling of social numbers has something conspicuously bureaucratic about it. And socialism was no stranger to the attempt to steer the economy by means of fictive calculations. Perhaps exactly here, there is empirical evidence that undermines the contrast between the market and the economic plan. The research project of the sociologist and accounting scholar Andrea Mennicken (London School of Economics) focuses on the performance-measurement of penitentiaries; in Great Britain, ever since the 1980s, they too have been obliged to conform to “New Public Management.” Back then, in the feeling that correctional facilities were being poorly administered by the state apparatus, it was mainly private security corporations which successfully lobbied to have correctional facilities be granted the status of firms.

In 1992 this led to the first privately run British prison. And here too, in the performance-measurement of prisons, one sees that typical blend of market rhetoric (competition, efficiency, return on investment) and intensive classificatory and bureaucratic work. Points systems for violations of prison regulations were developed that would add up to quality parameters; but these numbers not only hid the reality of the various prisons but the relative achievements of the concrete organizations. 

Must this be the fate of modern societies? "Decisions taken on the basis of numbers that all or most of the participants know to be inadequate insofar as they provide no reliable access to the particular reality that will be impacted by any decision? One might well assert that we have nothing better at our disposal and that questionable numbers are preferable to mere opinions, to sheer caprice. One might also contend that we lack the confidence to take responsibility for potentially bad decisions that have been taken without benefit of fictive numbers or at least in the face of them. Here too could be deployed one’s own research in asserting that social numbers are constantly ignored. For instance politicians have their own reasons for availing themselves of indicators – or not. But this only confirms the fact that an understanding of our present age is impossible without an explanation for our ambivalent dependence on numbers that we basically disbelieve.  

Next to the four fellows mentioned in the text, the focus group consists of another four scholars and their projects:  

Jahnavi Phalkey, a historian of science at King’s College London, studies the development of productivity measures associated with state-formation in India.

John Carson is a historian at the University of Michigan. He is researching the concept of “unsoundness of mind” as it developed between Anglo-American medicine and law during the eighteenth and nineteenth centuries.  

Tong Lam, also a historian (University of Toronto), examines the development of China's science and technology parks as a new form of utopian space for production and innovation in the post socialist era. He analyzes, as well, how numerical resoning continues to play a vital role in creating new citizen-workers, as well as imagining and ordering these new spaces for China's high-speed growth.  

Lorraine Daston, a historian of science who teaches in Berlin and Chicago and is a Permanent Fellow of the Wissenschaftskolleg, is presently writing a book on the practices of natural science compendia in the nineteenth century. 

Governance and Numbers: Credit, Crime and Growth
Video recording of the Evening Colloquium at the Wissenschaftskolleg, February 19, 2014

In the photos from left to right: John Carson, Bruce Carruthers, Emmanuel Didier, Lorraine Daston, Theodore Porter, Tong Lam, Wendy Espeland, Jahnavi Phalkey 
Andrea Mennicken had not yet arrived at the time of the photo session.