Kenneth Silber is a New Jersey-based writer on politics, economics, history and science. He has written for FrumForum and is a senior editor at Research magazine. He blogs at Quicksilber.
Jim Manzi’s Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society is an intriguing investigation of the power, limits, and varieties of empirical knowledge. The book ranges widely across issues in the sciences, social sciences and business, before reaching some conclusions about how government policies might be improved by a greater reliance on careful experimentation.
Manzi is a software entrepreneur who has made a mark in conservative commentary in recent years at National Review and the Manhattan Institute. This complex book puts him in a tradition, with Friedrich Hayek and Sir Karl Popper, that emphasizes limits to knowledge of how societies work and thus views government intervention and centralization with wariness. I will return to Manzi’s political arguments, but I think a substantial part of Uncontrolled’s value is in its sharp thinking about how various disciplines seek reliable knowledge.
Physics and chemistry have made great headway, in major part, because they study phenomena that are relatively simple and suitable for controlled laboratory experiments. Astronomy similarly benefits from its subject’s low “causal density,” in Manzi’s apt phrase, with the natural conditions of isolated celestial bodies allowing repeated observations that approximate lab experiments (plus the field has tight linkage to physical laws that have been corroborated experimentally).
All sciences face the “problem of induction” or uncertainties that arise in generalizing from observations. But the successes of the “hard” sciences in making predictions and spawning technologies leave many of their findings hard to dispute. “I suppose that apples might start to rise tomorrow,” the biologist Stephen Jay Gould once wrote, “but the possibility does not merit equal time in physics classrooms.”
Causal density and what Manzi calls “holistic integration” (the parts of a system affecting each other) pose growing challenges as one moves away from physics and chemistry into biology and, more so, into fields (including some portions of biology) that grapple with human minds and behavior. “In certain respects, the so-called soft social sciences deal with the hardest problems,” he notes cogently.
A key response to such complexity has been the randomized field trial or RFT (also known by other names, such as randomized control trial), which arose in biomedical research and has spread into social sciences and the business world. Manzi, who founded a company that makes software expediting RFTs, is an enthusiast of this empirical approach, and rightly so. Randomized trials have considerable benefits in minimizing statistical quirks and investigator biases. However, they are expensive and not always practical, and so there is a countervailing trend, in biomedical research and beyond, of relying on cheaper but less rigorous number-crunching.
Manzi points out weaknesses in such non-experimental statistical modeling. For instance, a statistical analysis claiming that legalized abortion caused a significant drop in crime, which appeared in academic literature before being popularized by the book Freakonomics, had results that depended heavily on particular data sets and assumptions. Fixing one technical error made the effect disappear (only to be resurrected with a new data set tweaked differently), leaving an unresolved debate about whose choices of data and technique made more sense.
The epistemological take-away of Uncontrolled could be stated as follows: In evaluating claims made by scientists and other experts, give much more weight to whether and how the claims are supported by controlled lab experiments and/or randomized field trials. These are the gold standards of various types of empirical research, though they do not always apply and never resolve all uncertainties.
That lesson is a useful one for helping sort through the blizzard of expert assertions that perpetually generate much public confusion. It is also of value to see a conservative commentator grappling knowledgeably with scientific philosophy and methodology at a time when the “anti-science” conservative has become a liberal stereotype that, sadly, has some basis in fact.
Drawing out some political implications in the book’s final section, Manzi argues for greater experimentation to inform government policymaking. His proposals include granting federal waivers to maximize the leeway states have in spending funds and setting rules, and creating a new federal agency to promote empirical policy analysis. He suggests leaning toward the status quo when evidence is lacking on the effects of a proposed government intervention and that policy actions be done on a small scale first whenever possible.
There are limits to how far such ideas can be taken, as Manzi acknowledges. Some problems resist piecemeal trial-and-error, such as large-scale threats from an external aggressor, environmental crisis or rogue asteroid. He advocates “liberty-as-means,” where local experiments might include, say, banning box stores or pornography, rather than a “liberty-as-goal” proscription of all such restrictions. He adds that liberty-as-means must observe limits, including a right to exit any locality.
Worried about a tradeoff between economic innovation and social cohesion, as creative destruction creates embittered losers as well as winners, Manzi contends there “should be no long-term libertarian goal of eliminating the welfare system, but rather of ensuring it achieves its purpose with minimum negative side effects.”
The general tone of Uncontrolled is understated, as Manzi avoids overselling his arguments. He suggests in the book’s introduction that greater experimentation could “somewhat improve the rate of development of social science; somewhat improve our decisions about what social programs we choose to implement; and somewhat improve our overall political economy.” These assertions seem quite reasonable. Uncontrolled offers useful advice for navigating a hard-to-know world.
