Pushkinskaya st. 43. office 10
Rostov-on-Don, Russia
e-mail: info@hjournal.ru 
tel. +7(863) 269-88-14

cubsEN (2)

Enlarged Empirical Economics and the Quest for Validity: Facing the Ontological Intricacies of the Social Domain

Enlarged Empirical Economics and the Quest for Validity: Facing the Ontological Intricacies of the Social Domain

Journal of Institutional Studies, , Vol. 11 (no. 1),

This article focuses on the impact that the recent widening of empirical economics has on the quest for validity in this field. We begin by summarizing the continuous evolution from a primarily deductive economics to a more empirical one, especially emphasizing the broader experimental and survey-based evidence. Although these developments pave the way for an economics with greater empirical support, they also bring into this field the same validity concerns that mainstream economists naively thought to be avoidable (i. e., concerns with the external validity of experiments and with “test validity” issues largely addressed in other social sciences). We show how, ultimately, such developments force economists to confront some serious challenges and limitations in the quest for validity arising from four ontological peculiarities of the social domain: 1) the awareness of the inquiry on the part of the subject being studied; 2) the lack of relevant structural homogeneity between individuals’ shared psychological properties; 3) actions holistic dependence on the individuals’ complete past; and 4) the variable and holistic nature of cultural, conventionally mediated forms of interaction. We finally argue that openly acknowledging these problems would help economists to tone down their scientificity claims and avoid pseudo-scientific practices like endorsing assumptions refuted by experience.

Keywords: external validity; experimental economics; test validity; survey research; pseudoscience

  • Aaskoven, L. (2016). Fiscal Transparency, Elections and Public Employment: Evidence from the OECD. Economics & Politics, 28(3), 317–341.
  • Alm, J., McClelland, G. H. and Schulze, W. D. (1992). Why do people pay taxes? Journal of Public Economics, 48(1), 21–38.
  • Alt, J. E. and Lassen, D. D. (2006). Fiscal transparency, political parties, and debt in OECD countries. European Economic Review, 50(6), 1403–1439.
  • Angrist, J., Azoulay, P., Ellison, G., Hill, R. and Lu, S. F. (2017). Economic Research Evolves: Fields and Styles. American Economic Review, 107(5), 293–97.
  • Angrist, J. D. and Pischke, J.-S. (2010). The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics. Journal of Economic Perspectives, 24(2), 3–30.
  • Backhouse, R. E. (2010). The Puzzle of Modern Economics: Science or Ideology? Cambridge: Cambridge University Press.
  • Backhouse, R. and Cherrier, B. (2017). The age of the applied economist: the transformation of economics since the 1970s. History of Political Economy, 49, Supplement), 1–33.
  • Ball, S., Eckel, C., Grossman, P. J. and Zame, W. (2001). Status in markets. Quarterly Journal of Economics, 116(1), 161–88.
  • Bardsley, N. (2005). Experimental Economics and the Artificiality of Alteration. Journal of Economic Methodology, 12(2), 239–251.
  • Bertrand, M. and Mullainathan, S. (2001). Do people mean what they say? Implications for subjective survey data. American Economic Review, 91(2), 67–72.
  • Biddle, J. E. and Hamermesh, D. S. (2017). Theory and Measurement: Emergence, Consolidation, and Erosion of a Consensus. History of Political Economy, 49, Supplement, 34–57.
  • Blinder, A. S. (1991). Why are prices sticky? Preliminary results from an interview study. American Economic Review, 81(2), 89–96.
  • Boettke, P. J., Leeson, P. T. and Smith, D. J. (2008). The Evolution of Economics: Where We are and How We Got Here. The Long Term View, 7(1), 14–22.
  • Bond, T. N. and Lang, K. (2018). The Sad Truth About Happiness Scales: Empirical Results. NBER Working Papers, 24853. Cambridge, MA: National Bureau of Economic Research (NBER).
  • Bond T. N. and Lang K. (2019). The Sad Truth About Happiness Scales. Journal of Political Economy, forthcoming.
  • Borsboom, D., Cramer, A. O. J., Kievit, R. A., Scholten ,A. Z. and Franic, S. (2009). The End of Construct Validity. In: The Concept of Validity: Revisions, New Directions, and Applications, ed. by R. W. Lissitz. Charlotte, NC: Information Age Publishing, 135–170.
  • Boulier, B. L. and Goldfarb, R. S. (1998). On the use and nonuse of surveys in economics. Journal of Economic Methodology, 5(1), 1–21.
  • Boumans, M. (2016). Methodological ignorance: A comment on field experiments and methodological intolerance. Journal of Economic Methodology, 23(2), 139–146.
  • Bradler, C. (2015). How Creative Are You? – An Experimental Study on Self-Selection in a Competitive Incentive Scheme for Creative Performance. ZEW Discussion Papers, 15-021. Mannheim: Centre for European Economic Research (ZEW).
  • Campbell, D. T. (1957). Factors relevant to the validity of experiments in social settings. Psychological Bulletin, 54(4), 297–312.
  • Card, D., Chetty, R., Feldstein, M. and Saez, E. (2010). Expanding Access to Administrative Data for Research in the United States. NSF SBE 2020 White Paper. Arlington, VA: National Science Foundation Directorate of Social, Behavioral, and Economic Sciences.
  • Carpenter, J., Connolly, C. and Myers, C. K. (2008). Altruistic behavior in a representative dictator experiment. Experimental Economics, 11(3), 282–298.
  • Chetty, R. (2012). Time Trends in the Use of Administrative Data for Empirical Research. NBER Summer Institute presentation. Available at the author’s website.
  • Claveau, F. (2011). Evidential variety as a source of credibility for causal inference: beyond sharp designs and structural models. Journal of Economic Methodology, 18(3), 233–253.
  • Cubitt, R. (2005). Experiments and the domain of economic theory. Journal of Economic Methodology, 12(2), 197–210.
  • Curtin, R. T. (2004). Psychology and Macroeconomics. In: A Telescope on Society: Survey Research and Social Science at the University of Michigan and Beyond, ed. by J. S. House, F. T. Juster, R. L. Kahn, H. Schuman and E. Singer. Ann Arbor, MI: University of Michigan Press, 121–155.
  • Deaton, A. and Stone, A. A. (2016). Understanding context effects for a measure of life evaluation: how responses matter. Oxford Economic Papers, 68(4), 861–870.
  • Dohmen, T. and Falk, A. (2011). Performance Pay and Multidimensional Sorting: Productivity, Preferences, and Gender. American Economic Review, 101(2), 556–590.
  • Downward, P. and Mearman, A. (2007). Retroduction as mixed-methods triangulation in economic research: reorienting economics into social science. Cambridge Journal of Economics, 31(1), 77–99.
  • Einav, L. and Levin, J. (2014). Economics in the age of big data. Science, 346(6210), 1243089.
  • Falk, A., Becker, A., Dohmen, T., Enke, B., Huffman, D. and Sunde, U. (2018). Global evidence on economic preferences. The Quarterly Journal of Economics, 133(4), 1645–1692.
  • Falk, A., Becker, A., Dohmen, T., Huffman, D. and Sunde, U. (2016). The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences. IZA Discussion Papers, 9674. Bonn: Institute for the Study of Labor (IZA).
  • Fudenberg, D. (2006). Advancing Beyond Advances in Behavioral Economics. Journal of Economic Literature, 44(3), 694–711.
  • Graham, C. (2008). The Economics of Happiness. In: The New Palgrave Dictionary of Economics, 2nd edition, ed. by S. Durlauf and L. Blume. Hampshire: Palgrave MacMillan.
  • Greenwood, J. D. (1982). On the relation between laboratory experiments and social behaviour: causal explanation and generalisation. Journal of the Theory of Social Behaviour, 12(3), 225–250.
  • Groves, R. M. and Singer, E. (2004). Survey Methodology. In: A Telescope on Society: Survey Research and Social Science at the University of Michigan and Beyond, ed. by J. S. House, F. T. Juster, R. L. Kahn, H. Schuman and E. Singer. Ann Arbor, MI: University of Michigan Press), 21–64.
  • Guala, F. (2003). Experimental Localism and External Validity. Philosophy of Science, 70(5), 1195–1205.
  • Guala, F. (2005). The Methodology of Experimental Economics, Cambridge: Cambridge University Press.
  • Guala, F. (2008). Experimentation in Economics. 2nd draft, prepared for the Elsevier Handbook of the Philosophy of Science, 13: Philosophy of Economics, ed. by U. Mäki. (http://users.unimi.it/guala/Handbook%20Elsevier3.pdfusers.pdf – Access Date: 09.01.2019).
  • Hamermesh, D. S. (2013). Six Decades of Top Economics Publishing: Who and How? Journal of Economic Literature, 51(1), 162–172.
  • Harrison, G. W. and List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.
  • Hausman, D. (1992). The Inexact and Separate Science of Economics. Cambridge: Cambridge University Press.
  • Hausman, D. M. (2018). Philosophy of Economics. In: The Stanford Encyclopedia of Philosophy (Fall 2018 Edition), ed. by Edward N. Zalta. (https://plato.stanford.edu/archives/fall2018/entries/economics/ – Access Date: 09.01.2019).
  • Heckman, J. J. (2000). Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective. Quarterly Journal of Economics, 115(1), 45–97.
  • Heckman, J. J. (2005). The Scientific Model of Causality. Sociological Methodology, 35(1), 1–97.
  • Heckman, J. J. (2008). Econometric Causality. International Statistical Review, 76(1), 1–27.
  • Heukelom, F. (2011). How validity travelled to economic experimenting. Journal of Economic Methodology, 18(1), 13–28.
  • Hoffman, E., McCabe, K. and Smith, V. L. (1996). Social Distance and Other-Regarding Behavior in Dictator Games. The American Economic Review, 86(3), 653–660.
  • Hogarth, R. M. (2005). The challenge of representative design in psychology and economics. Journal of Economic Methodology, 12(2), 253–263.
  • Holländer, H. (2001). On the validity of utility statements: standard theory versus Duesenberry’s. Journal of Economic Behavior and Organization, 45(3), 227–249.
  • Hutchison, T. (1998). Ultra-deductivism from Nassau Senior to Lionel Robbins and Daniel Hausman. Journal of Economic Methodology, 5(1), 43–91.
  • Imbens, G. W. and Wooldridge, J. M. (2009). Recent Developments in the Econometrics of Program Evaluation. Journal of Economic Literature, 47(1), 5–86.
  • Jensen, N. M., Mukherjee, B. and Bernhard, W. T. (2014). Introduction: Survey and Experimental Research in International Political Economy. International Interactions, 40(3), 287–304.
  • Jiménez-Buedo, M. (2011). Conceptual tools for assessing experiments: some wellentrenched confusions regarding the internal/external validity distinction. Journal of Economic Methodology, 18(3), 271–282.
  • Jimenez-Buedo, M. and Guala, F. (2016). Artificiality, Reactivity, and Demand Effects in Experimental Economics. Philosophy of the Social Sciences, 46(1), 3–23.
  • Jiménez-Buedo, M. and Miller, L. M. (2010). Why a Trade-Off? The Relationship between the External and Internal Validity of Experiments. THEORIA. An International Journal for Theory, History and Foundations of Science, 25(3), 301–321.
  • Jones, M. K. (2007). A Gricean analysis of understanding in economic experiments. Journal of Economic Methodology, 14(2), 167–185.
  • Juster, F. T. (2004). The Behavioral Study of Economics. In: A Telescope on Society: Survey Research and Social Science at the University of Michigan and Beyond, ed. by J. S. House, F. T. Juster, R. L. Kahn, H. Schuman and E. Singer. Ann Arbor, MI: University of Michigan Press, 119–130.
  • Kahneman, D. and Krueger, A. B. (2006). Developments in the Measurement of Subjective Well-Being. The Journal of Economic Perspectives, 20(1), 3–24.
  • Kalaitzidakis, P., Mamueas, T. P. and Stengos, T. (2003). Rankings of academic journals and institutions in economics. Journal of the European Economic Association, 1(6), 1346–1366.
  • Kling, C. L., Phaneuf, D. J. and Zhao, J. (2012). From Exxon to BP: Has Some Number Become Better Than No Number? Journal of Economic Perspectives, 26(4), 3–26.
  • Krueger, A. B. and Stone, A. A. (2014). Progress in measuring subjective well-being: Moving toward national indicators and policy evaluations. Science, 346(6205), 42–43.
  • Lawson, T. (2015a). A Conception of Social Ontology. In: Social Ontology and Modern Economics, ed. by S. Pratten. London: Routledge, 19–52.
  • Lawson, T. (2015b). Methods of abstraction and isolation in modern economics. In: Social Ontology and Modern Economics, ed. by S. Pratten. London: Routledge, 315–337.
  • Levitt, S. D. and List, J. A. (2009). Field experiments in economics: The past, the present, and the future. European Economic Review, 53(1), 1–18.
  • Lissitz, R. W. (ed.) (2009). The Concept of Validity: Revisions, New Directions, and Applications. Charlotte, NC: Information Age Publishing.
  • List, J. A. (2009). An introduction to field experiments in economics. Journal of Economic Behavior & Organization, 70(3), 439–442.
  • Loewenstein, G. (1999). Experimental Economics from the Vantage-Point of Behavioural Economics. The Economic Journal, 109(453), 25–34.
  • Lucking-Reiley, D. (1999). Using field experiments to test equivalence between auction formats: Magic on the internet. American Economic Review, 89(5), 1063–1080.
  • Mäki, U. (2018). Empirical turn in economics? The 30th Annual EAEPE (European Association for Evolutionary Political Economy) Conference 2018. Evolutionary foundations at a crossroad: Assessments, outcomes and implications for policy makers, Nice, France, 06–08 September.
  • Markus, K. A. and Borsboom, D. (2013). Frontiers of Test Validity Theory: Measurement, Causation, and Meaning. New York: Routledge.
  • Mason, C. F., Phillips, O. R. and Nowell, C. (1992). Duopoly behavior in asymmetric markets: an experimental evaluation. The Review of Economics and Statistics, 74(4), 662–670.
  • McCloskey, D. N. and Ziliak, S. T. (2001). Measurement and meaning in economics: the essential Deirdre McCloskey. Cheltenham, UK: Edward Elgar.
  • Meyer, B. D., Mok, W. K. C. and Sullivan, J. X. (2015). Household Surveys in Crisis. Journal of Economic Perspectives, 29(4), 199–226.
  • Morgan, J. and Patomäki, H. (2017). Contrast explanation in economics: its context, meaning, and potential. Cambridge Journal of Economics, 41(5), 1391–1418.
  • Morgan, J. N. (1967). Contributions of survey research to economics. In: Survey Research in the Social Sciences, ed. by C. Glock. New York: Russell Sage Foundation, 217–268.
  • Naef, M. and Schupp, J. (2009). Measuring Trust: Experiments and Surveys in Contrast and Combination. IZA Discussion Papers, 4087. Bonn: Institute for the Study of Labor (IZA).
  • Newman, J. (2017). Our Obsession with Survey Data is Ruining Economics. Article posted at July 23, 2017 on the Mises Wire. (https://mises.org/blog/our-obsession-survey-dataruining-economics – Access Date: 09.01.2019)
  • Panhans, M. T. and Singleton, J. D. (2017). The empirical economist’s Toolkit: from models to methods. History of Political Economy, 49, Supplement, 127–157.
  • Perraton, J. (2011). Explaining growth? The case of the trade-growth relationship. Journal of Economic Methodology, 18(3), 283–296.
  • Porter, M. E., Ketels, C. and Delgado, M. (2007). The Microeconomic Foundations of Prosperity: Findings from the Business Competitiveness Index. In: The Global Competitiveness Report 2007–2008, ed. by M. E. Porter, K. Schwab, and X. Sala-i-Martin. London: Palgrave Macmillan, 51–81.
  • Powdthavee, N. (2007). Economics of Happiness: A Review of Literature and Applications. Chulalongkorn Journal of Economics, 19(1), 51–73.
  • Powdthavee, N. (2015), Would You Like to Know What Makes People Happy? An Overview of the Datasets on Subjective Well-Being. The Australian Economic Review, 48(3), 314–320.
  • Presser, S. (1984). The Use of Survey Data in Basic Research in the Social Sciences. In: Surveying Subjective Phenomena, ed. by C. F. Turner and E. Martin. New York: Russell Sage Foundation, 93–114.
  • Presser, S. and McCullogh, S. (2011). The Growth of Survey Research in the United States: Government-Sponsored Surveys, 1984–2004. Social Science Research, 40(4), 1019–1024.
  • Price, L. (2017). Psychometric Methods: Theory into Practice. New York: The Guilford Press.
  • Read, D. (2005). Monetary incentives, what are they good for? Journal of Economic Methodology, 12(2), 265–276.
  • Rodrick, D. (ed.) (2003). In Search for Prosperity. Analytical Narratives on Economic Growth. Princeton: Princeton University Press.
  • Rosenberg, A. (1992). Economics: Mathematical Politics or Science of Diminishing Returns? Chicago: University of Chicago Press.
  • Rosenberg, A. (2009). If Economics is a Science, What Kind of a Science Is It? In: The Oxford Handbook of Philosophy of Economics, ed. by H. Kincaid and D. Ross. Oxford: Oxford University Press, 55–67.
  • Rosenberg, A. (2015). Philosophy of Social Science, 5th edition. Boulder, CO: Westview Press.
  • Ross, D. (2010). Why economic modelers can’t exclude psychological processing variables. Journal of Economic Methodology, 17(1), 87–92.
  • Ross, D. (2011). Estranged parents and a schizophrenic child: choice in economics, psychology and neuroeconomics. Journal of Economic Methodology, 18(3), 217–231.
  • Ross, D. and Kincaid, H. (2009). Introduction: The New Philosophy of Economics. In: The Oxford handbook of philosophy of economics, ed. by H. Kincaid and D. Ross, Oxford: Oxford University Press, 3–32.
  • Ruzzene, A. (2014). Using case studies in the social sciences. Methods, inferences, purposes. Thesis to obtain the degree of Doctor from the Erasmus University Rotterdam (EUR) bycommand of the rector magnificus Prof. Dr. H. A. P. Pols and in accordance with the decision of the Doctorate Board. Rotterdam: EUR.
  • Santos, A. C. (2009). Behavioral experiments: how and what can we learn about human behavior. Journal of Economic Methodology, 16(1), 71–88.
  • Saris, W. E. and Gallhofer, I. N. (2007). Design, Evaluation, and Analysis of Questionnaires for Survey Research. Hoboken, New Jersey: John Wiley & Sons.
  • Schotter, A. (2008). What’s So Informative about Choice? In: The foundations of positive and normative economics: a handbook, ed. by A. Caplin and A. Schotter. Oxford: Oxford University Press, 70–94.
  • Schram, A. (2005). Artificiality: The tension between internal and external validity in economic experiments. Journal of Economic Methodology, 12(2), 225–237.
  • Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton-Mifflin.
  • Shulman, R. G. and Shapiro, I. (2009). Reductionism in the Human Sciences: A Philosopher’s Game. In: Philosophy of the social sciences: philosophical theory and scientific practice, ed. by C. Mantzavinos. Cambridge: Cambridge University Press, 124–129.
  • Siakantaris, N. (2000). Experimental Economics Under the Microscope. Cambridge Journal of Economics, 24(3), 267–281.
  • Siedler, T. and Sonnenberg, B. (2010). Experiments, Surveys and the Use of Representative Samples as Reference Data. RatSWD Working Papers, 146. Berlin: Rat für Sozial- und Wirtschaftsdaten (RatSWD).
  • Simon, H. A. (1997). An Empirically Based Microeconomics. Cambridge: Cambridge University Press.
  • Smith, V. L. (2010). Theory and experiment: What are the questions? Journal of Economic Behavior & Organization, 73(1), 3–15.
  • Suppes, P. (1982). Problems of Causal Analysis in the Social Sciences. Epistemologia. Rivista Italiana di Filosofia della Scienza, 5, special issue, 239–250.
  • Syll, L. P. (2016). Deductivism: the fundamental flaw of mainstream economics. Real World Economics Review, (74), 20–41.
  • Venaik, S. (2007). Abstract of the seminar: Factors affecting the choice of measurement models in SEM. Teached at the NUS on April 17. (http://bschool.nus.edu.sg/Departments/BussPolicy/Seminar%20Abstract_%20Profiles/SunilVenaik.abstract.htm – Access Date: 20.07.2011)
  • Vischer, T., Dohmen, T., Falk ,A., Huffman, D., Schupp, J., Sunde, U. and Wagner, G. G. (2013). Validating an Ultra-Short Survey Measure of Patience. Economics Letters, 120(2), 142–145.
  • Voigt, S. (2013). How (Not) to measure institutions. Journal of Institutional Economics, 9(1), 1–26.
  • Vugt, M. V. (2001). Self-interest as self-fulfilling prophecy. Behavioral and Brain Sciences, 24(3), 429–430.
  • Wainer, H. and Braun, H. I. (eds.) (1988). Test validity. Hillsdale, NJ: Lawrence Erlbaum Associates.
  • Wiggins, J. A. (1968). Hypothesis Validity and Experimental Laboratory Methods. In: Methodology in Social Research, ed. by H. M. Blalock, Jr., and A. B. Blalock. New York: McGraw-Hill, 390–427.
Publisher: Ltd. "Humanitarian perspectives"
Founder: Ltd. "Humanitarian perspectives"
Online-ISSN: 2412-6039
ISSN: 2076-6297