본문 바로가기

[연구] Research

[IR-Theories Evidences Inferences] Fearon, James (1991) “Counterfactuals and Hypothesis Testing in Political Science”

Fearon, James (1991) “Counterfactuals and Hypothesis Testing in Political Science,” World Politics, Vol. 43, January 1991, pp. 169-195.

In making a causal claim, that C was the cause of event E, Fearon suggests here that there are two empirical strategies: by making 1) a counterfactual case (~ if it had not been the case that C (or not C), it would have been the case that E (or not E); or 2) a comparative approach (comparison of actual cases). While counterfactual case remains underdeveloped and underacknowledged in political science (for its “speculative” approach), Fearon argues that many overlook the fact that the comparison of actual cases also requires counterfactual thoughts for serious justification and elaboration of causal claims: Even if comparison of actual cases provides us the existing samples to compare and therefore the seemingly more degree of references (evidences and therefore confidence), the process also requires counterfactual process in picking the right variables and justifying that these cases are appropriately comparable/identical… and this is more so in small-N research where there are too many variables and too few cases.


NOTE

Counterfactuals: If not C, then not E

(“If it had been the case that C(or not C), it would have been the case that E(or not E).”(169).

Fearon: such propositions “play a necessary and fundamental, if often implicit and underdeveloped, role in the efforts of political scientists to assess their hypotheses about the causes of the phenomena they study” (169)

Focus on “the role of counterfactuals in small-N research” (174) – “the necessity of counterfactual argument for justifying causal claims in small-N settings… the point is that when degrees of freedom in the actual world are negative, a causal claim requires argument about counterfactual cases for its justification (or addition of other actual cases).” (180)

 

Main arguments:

1)      Counterfactual propositions and arguments play “a central role in the efforts of political scientists to assess their causal hypotheses” (170) – examples: i) on the causes of WWI; ii) the nonoccurrence of events such as WWIII, social revolutions, the breakdown of democratic regimes in Latin America; and iii) the origin of fascist and corporatist regimes in Latin America

2)      Counterfactual method vs. Comparison of the actual cases: This strategy is “related but also differs from methods of hypothesis testing based on the comparison of the actual cases”(170)

3)      To address the question: “Is counterfactual argument a viable means of assessing causal hypotheses in nonexperimental research settings?”(170)

 

Structure:

1)      Section I: Outlining the differences between the two strategies of hypothesis testing: i) comparison of actual cases; and ii) counterfactual argument

2)      Section II: Showing examples of counterfactual cases in international relations and comparative politics

3)      Section III: Questioning the viability of the counterfactual strategy

 

Counterfactuals, Actual Case Comparisons, and the Logic of Inference

>Hypothesis: C was the cause of event E<

ð  To test this hypothesis, according to Fearon, there are “only two strategies” for “empirically”(171) assessing this hypothesis, which aim to “solve the same statistical problem” (173).

1)      Counterfactual case: Imagine C had been absent and ask whether E would have been possible…

2)      Comparison of actual cases: that resemble E but where C is sometimes absent or had different value =  testing the association between the occurrence of C & E in the set of actual cases = formally known as the regression analysis

ð  Also, we cannot but explain why some event E occurred rather than some other possible outcomes.

Note on the main risks with these strategies:

1)      Counterfactual case: “how can we know what would have happened with any degree of confidence?”(173) – a question avoided by many historians (“historian should never deal in speculations”) and also political scientists and sociologists who preferred to deal with actual cases and refrain from the question

-          Exception:

n  Weber (1949) “Objective Possibility and Adequate Causation in Historical Explanation,” in The Methodology of the Social Sciences, Free Press

n  Elster (1978) Logic and Society: Contradictions and Possible Worlds, Wiley.

n  Also addressed: Nelson Polsby (1982) What If?: Essays in Social Science Fiction, Lewis Publishing.  

2)      Comparison of actual cases: Even if the degrees of freedom have been increased (including more actual cases), how can we know “if the additional cases are appropriately identical”(173).

 

Case application:

1)      The Cause of WWI: by a “cult of the offensive” and belief in the advantage of striking first:

-          Actual cases comparison: set of international disputes that some escalated to war and some that did not => variables control => test for the association between “commitment to offensive doctrines and escalation”(176)

-          Counterfactual case strategy: “which often goes under the name “case study””(177) – “imagine the prewar world without a cult of the offensive but otherwise similar” (177) -> and show that “the outbreak of a general war would have been much less likely” in this counterfactual case – e.g. Stephen Van Evera who asked “How would statesmen have behaved if they had believed that defense rather than offense had the advantage?” (177)

 

Differences yet Similarities between the Two Strategies:

1)      Difference – dependence on other theories:

-          Counterfactual, to make its case, depends on “invoking others – laws, regularities, or principles” (177) as in Van Evera’s dependence on rationality

-          Actual case strategy – no need of “other principles,” “only a strength of association across actual cases matters” (177)

2)      Similarity: actual case strategy itself implicitly depends on counterfactuals to be confident that “the other causes would not vary” as well…

-          “when the actual case strategy is employed in a nonexperimental setting, the validity of a causal interpretation of the results in contingent on the truth of a counterfactual assumption about the other unspecified, unmeasured causes. We must be ready to accept the proposition that had variable X taken values different from those in the sample, no such other causes of the dependent variable would have been systematically different as well” (177)

3)      More differences:

-          Causal weight: actual cases can extract contrasting weights from the sample (frequencies and magnitudes can be measured); whereas counterfactuals lack in the sample for varying weight -> inevitably, “a proliferation of counterfactual cases”

-          Precision of estimates: Counterfactuals lack in criterion for gauging the risk of error associated with some independent variable. All depends on the plausibility of arguments about what would have happened.

 

Counterfactuals and Causation

 

Conclusion

“Counterfactuals and the counterfactual strategy of hypothesis testing play an important but often unacknowledged and underdeveloped role in the efforts of political scientists to assess causal hypotheses… any non experimental research that makes causal claims, be it of the large-N or small-N variety, must confront counterfactuals in the form of key assumptions or in the use of hypothetical comparison cases. Particularly in small-N research, the common condition of too many variables and too few cases makes counterfactual thought experiments a necessary means for serious justification of causal claims.” (p. 194)