Research method

Some reflections on scientific method

Economics has occasionally been described as 'more art than science'. I think academic economists can become quite agitated by this characterization, but there is a certain element of truth. First, empirical economics is far removed from being an experimental science. Consequently, the influence of alternative variables and mechanisms not being controlled for will always create debate and opportunities for alternative interpretations of empirical results. Second, empirical research is subject to various errors (measurement errors, methodological errors or simply errors of repetition - flawed statements or conclusions that are just mindlessly repeated by referral to previously published work) and theoretical research based on mathematical models is sometimes very sensitive to small changes in initial assumptions. I believe that the 'art' in economic science refers to the idea that statements, conclusions, empirical results in the economic literature must always be carefully evaluated (again-and-again) and cannot be taken at face value alone; unfortunately. Selecting the correct empirical results and the appropriate theoretical analysis from a wide range range of mixed results and contradictory alternatives is truly an art, rather than science.

A scientist once trained a flea to jump whenever he rang a bell. Using a microscope, he then anesthetized one of the flea's legs and rang the bell again. The flea still jumped.
The scientist then anesthetized another leg and then rang the bell. The flea still jumped.
Eventually, the scientist anesthetized more and more legs, each time ringing the bell, and each time recording that the flea jumped.
Finally, the flea had only one leg left. When the scientist anesthetized the last leg and rang the bell, he found to his surprise that the flea no longer jumped.
Then the scientist solemnly declared his conclusion, based on irrefutable scientific data: Fleas hear through their legs!

Michio Kaku, Hyperspace: A Scientific Odyssey Through Parallel Universes, Time Warps, and the 10th Dimension. New York: Oxford University Press, 1994 (p. 118)

Whenever an experimental scientist achieves a substantial empirical breakthrough, his colleagues' first impulse is to try the experiment for themselves. Scientists are skeptical men who know that errors of observation and questionable experimental technique are facts of life. If a law verified by one observer is truly valid, it should operate for anyone. Successful and independent repetition of a critical experiment increases our confidence in its supposed results. By reducing the probability that a reported observation is due either to chance or to experimental error, replication can add significantly to the stock of scientific knowledge.

Malkiel and Kane (1969 JPE p.453)

[The] image of the applied econometrician's art is grossly misleading. I would like to suggest a more accurate one. The applied econometrician is like a farmer who notices that the yield is somewhat higher under trees where birds roost, and he uses this as evidence that bird droppings increase yields. However, when he presents this finding at the annual meeting of the American Ecological Association, another farmer in the audience objects that he used the same data but came up with the conclusion that moderate amounts of shade increase yields. A bright chap in the back of the room then observes that these two hypotheses are indistinguishable, given the available data.

Leamer (1983 AER p.31)

Research method

Some reflections on scientific method

Other relevant articles