Re: A Critique of Social Science (part II)

In this episode I'd like to consider what exactly is the complexity barrier and where it lies. It seems to be that the idea of the barrier is easily stated, but determining where it lies is less simple. The barrier marks the boundary between phenomenon which yield to rigorous scientific experimentation and those too complex for experimenters to isolate significant factors or where no experimentation can be done. Henry is basically correct in placing the social sciences on the untestable side. V argues that this characterization is overinclusive and that there are some meaningfully testable social science findings. I'm willing to buy his premise, but I don't know of any particularly good examples, so I'll leave that discussion for him.

However, I think that while this barrier tells us something important about the social sciences, it does not stand as a wall between the social sciences and the physical sciences. Significant parts of the physical sciences also fall on the untestable side, or in the gray middle ground. The medical sciences serve as a particularly good example. Epidemiological studies resemble in many ways the sort of data obtained by social scientists, and come with many of the same caveats with regards to the difficulty of isolating significant factors. Even many smaller drug tests suffer these flaws, though to a smaller degree. Blind tests and control groups add a good measure of rigor to these tests, but cannot claim to isolate interfering factors in any absolute way. We don't completely understand the causal mechanics of many afflictions, drugs, and treatments, and can only measure their effects on people, an inexact process. Additionally the vast realm spanning neurology (a physical science) and psychology (a social science) is difficult to characterize decisively as testable or nontestable. Ryan may know something of this.. :)

Other physical sciences also encounter barriers of testability. String theory, for example, is a popular topic of physics, but has so far not yielded to any useful testing. Likewise, cosmology is an entire field built around untestable theories. Cosmologists gather immense amounts of data against which to compare the predicted results of their theories, but they remain unable to ever perform actual experimentation.

The matter of what qualifies as a scientifically rigorous test that sufficiently isolates relevant factors is something of a philosophical question. We can't know, even in a high school physics experiment on acceleration whether aliens are firing a gravity gun from orbit at our ball, skewing results, or whether the supposed constants of the universe vary over time, and perhaps today was just a heavy day in the universe. But we reasonably decide to dismiss concerns about such improbable factors and consider this experiment to be rigorously scientific. While my example is silly, I wonder how far this judgment extends and how we define the boundary.

It also is interesting to note that in many of these cases the barrier of testability shifts over time. MRI's, electron microscopes, particle colliders and other technological tools have shattered barriers of testability in many fields. In other cases all that might be needed is a new insight into the predicted effects of a theory (i.e. string theory) that would yield a testable phenomenon.

None of this is to suggest that the social sciences are any more rigorous than Henry stated. However, I think it is inaccurate to level this criticism only at social sciences as compared to the physical sciences. It is an important barrier, and an enhanced awareness of the barrier would be useful to people employed in the sciences, or interested in them. The truth value of a claim in a testable field is obviously different from the truth value of a claim in an untestable field, and it would be helpful if more people were cognizant of the distinction.