Saturday, February 6, 2010

The inexact science of economics

Image: social accounting matrix, Bolivia, 1997

Economics is an "inexact" science; or so Daniel Hausman argues in The Inexact and Separate Science of Economics (Google Books link).  As it implies, this description conveys that economic laws have only a loose fit with observed economic behavior.  Here are the loosely related interpretations that Hausman offers for this idea, drawing on the thinking of John Stuart Mill:
  1. Inexact laws are approximate.  They are true within some margin of error.
  2. Inexact laws are probabilistic or statistical.  Instead of stating how human beings always behave, economic laws state how they usually behave.
  3. Inexact laws make counterfactual assertions about how things would be in the absence of interferences.
  4. Inexact laws are qualified with vague ceteris paribus clauses. (128)
Economics has also been treated by economists as a separate science: a science capable of explaining virtually all the phenomena in a reasonably well-defined domain of social phenomena.  Here is Hausman's interpretation of a separate science:
  1. Economics is defined in terms of the causal factors with which it is concerned, not in terms of a domain.
  2. Economics has a distinct domain, in which its causal factors predominate.
  3. The "laws" of the predominating causal factors are already reasonably well-known.
  4. Thus, economic theory, which employs these laws, provides a unified, complete, but inexact account of its domain. (90-91)
These characteristics of economic theories and models have implications for several important areas: truth, prediction, explanation, and confirmation.  Is economics a scientific theory of existing observable economic phenomena?  Or is it an abstract, hypothetical model with only tangential implications for the observable social world?  Is economics an empirical science or a mathematical system?

Let's look at these questions in turn.  First, can we give a good interpretation of what it would mean to believe that an inexact theory or law is "true"?  Here is a possible answer: we may believe that there are real but unobservable causal processes that "drive" social phenomena.  To say that a social or economic theory is true is to say that it correctly identifies a real causal process -- whether or not that process operates with sufficient separation to give rise to strict empirical consequences.  Galilean laws of mechanics are true for falling objects, even if feathers follow unpredictable trajectories through turbulent gases.

Second, how can we reconcile the desire to use economic theories to make predictions about future states with the acknowledged inexactness of those theories and laws? If a theory includes hypotheses about underlying causal mechanisms that are true in the sense just mentioned, then a certain kind of prediction is justified as well: "in the absence of confounding causal factors, the presence of X will give rise to Y." But of course this is a useless predictive statement in the current situation, since the whole point is that economic processes rarely or never operate in isolation. So we are more or less compelled to conclude that theories based on inexact laws are not a useable ground for empirical prediction.

Third, in what sense do the deductive consequences of an inexact theory "explain" a given outcome -- either one that is consistent with those consequences or one that is inconsistent with the consequences? Here inexact laws are on stronger ground: after the fact, it is often possible to demonstrate that the mechanisms that led to an outcome are those specified by the theory. Explanation and prediction are not equIvalent. Natural selection explains the features of Darwin's finches -- but it doesn't permit prediction of future evolutionary change.

And finally, what is involved in trying to use empirical data to confirm or disconfirm an inexact theory?  Given that we have stipulated that the theory has false consequences, we can't use standard confirmation theory.  So what kind of empirical argument would help provide empirical evaluation of an inexact theory?  One possibility is that we might require that the predictions of the theory should fall within a certain range of the observable measurements -- which is implied by the idea of "approximately true" consequences.  But actually, it is possible that we might hold that a given theory is inexact, true, and wildly divergent from observed experience.  (This would be true of the application of classical mechanics to the problem of describing the behavior of light, irregular objects shot out of guns under water.)  Hausman confronts this type of issue when he asks why we should believe that the premises of general equilibrium theory are true. But here too there are alternatives, including piecemeal confirmation of individual causal hypotheses. Hausman refers to this possibility as a version of Mill's deductive method.

I take up some of these questions in my article, "Economic Models in Development Economics" link, included in On the Reliability of Economic Models: Essays in the Philosophy of Economics.  This article discusses some related questions about the reliability and applicability of computable general equilibrium models in application to the observed behavior of real economies.  Here are some concluding thoughts from that article concerning the empirical and logical features that are relevant to the assessment of CGE models:

"The general problem of the antecedent credibility of an economic model can be broken down into more specific questions concerning the validity, comprehensiveness, robustness, reliability, and autonomy of the model. I will define these concepts in the following terms.
  • Validity is a measure of the degree to which the assumptions employed in the construction of the model are thought to correspond to the real processes underlying the phenomena represented by the model.
  • Comprehensiveness is the degree to which the model is thought to succeed in capturing the major causal factors that influence the features of the behavior of the system in which we are interested.
  • Robustness is a measure of the degree to which the results of the model persist under small perturbations in the settings of parameters, formulation of equations, etc.
  • Autonomy refers to the stability of the model's results in face of variation of contextual factors.
  • Reliability is a measure of the degree of confidence we can have in the data employed in setting the values of the parameters.
These are epistemic features of models that can be investigated more or less independently and prior to examination of the empirical success or failure of the predictions of the model."

(Hausman's book is virtually definitive in its formulation of the tasks and scope of the philosophy of economics.  When conjoined with the book he wrote with Michael McPherson, Economic Analysis, Moral Philosophy and Public Policy, the philosophy of economics itself becomes a "separate science": virtually all the important questions are raised throughout a bounded domain, and a reasonable set of theories are offered to answer those questions.)

4 comments:

Tom Hickey said...

Thanks for the summary.

David Sloan Wilson has some interesting things to say about this from the perspective of evolution here. He is particularly critical of New Classicalism, especially as influenced by Robert Lucas, for being a mathematical system based on tenuous assumptions, hence, rather divorced from the real conditions it purports to model. It's a good short read in three parts.

Alex said...

Sounds like some of this would have some relation to the Bayes vs frequentist debate. Am I right?

Anonymous said...

Would you please help me understand the following?

"Given that we have stipulated that the theory has false consequences, we can't use standard confirmation theory. So what kind of empirical argument would help provide empirical evaluation of an inexact theory? One possibility is that we might require that the predictions of the theory should fall within a certain range of the observable measurements -- which is implied by the idea of "approximately true" consequences. But actually, it is possible that we might hold that a given theory is inexact, true, and wildly divergent from observed experience."

In that final sentence, do you mean to say that "while an economic theory can diverge wildly from observed experience, the way to evaluate it is to check it in relation a range of observed experience that we determine in advance to be relevant to it?"

Dan Little said...

I've probably packed too many ideas into a few sentences here. Here's how I might unpack it as a series of interpretations of how to test or confirm an economic theory:

1. Falsifiability (Popper): does the theory have any false consequences? If so, the theory is false; reject it.
2. Confirmation theory (Hempel): gather the observable deductive consequences of the theory. If a sufficient proportion are true then the theory is confirmed; accept it.
3. Adjust deductive consequences of the theory by specifying an estimated error range (for illustration, +/- 3%). If the bulk of observations fall within the error range, the theory is confirmed as approximately true. (This would be an effort to accommodate ceteris paribus conditions as minor disturbing factors.)

1 and 2 are disqualified immediately by the assumption of inexactness. We stipulate that the consequences of the theory are strictly false. But my final sentence is challenging the applicability of 3 as well when it comes to empirically evaluating economic theories: the "countervailing tendencies" or untheorized factors don't simply distort the behavior of the system by a small delta (approximation error), but instead lead to wide deviations of behavior.

I have a physical situation in mind: imagine modeling the trajectory of a bunch of bananas shot out of a cannon under water with strong currents and thermal layers. We can compute the trajectory based on the mechanics of the situation. But the actual flight path is influenced by the contingencies of irregular shape, low mass, and unpredictable currents and temperature shifts, with the result that the observed trajectory will differ wildly from the computed trajectory. But here is the crucial point: the laws of mechanics are still true in this circumstance; but contingencies, untheorized causal processes, and countervailing tendencies have distorted the actual behavior of the projectile beyond any notion of "approximation". If we were Galilean sea-dwelling scientists we wouldn't be able to test our theories of mechanics and motion using this kind of experiment.

What this implies for economic theory is taken up when I refer to the "deductive" method in opposition to the hypothetico-deductive method: find ways of separately evaluating the "truthiness" of the causal hypotheses one by one rather than as a deductive whole. Hausman discusses Mill's deductive method of confirmation in this book and in an earlier essay.