February 2016
Mon Tue Wed Thu Fri Sat Sun
« Jan   Mar »
1234567
891011121314
15161718192021
22232425262728
29  

Month February 2016

Why you shouldn’t use Twitter to predict elections

Yesterday the Social Science Computer Review published a new paper by Harald Schoen, Oliver Posegga, Pascal Jürgens, and me on the futility of trying to predict elections with Twitter. This claim shouldn’t be all that controversial, but apparently it is.

Quite a few studies have identified fundamental problems with Twitter-based predictions. Especially Daniel Gayo-Avello has written multiple times about systematic flaws in these attempts, so has Takis Metaxas, and Mark Huberty. More fundamentally Fernando Diaz and colleagues have identified central challenges in interpreting social media data as proxies for survey results because of irregular fluctuations in the services’ user base and difficulties to consistently map this activity to established sentiment or opinion measures. The predictive potential of other digital services has also been challenged. For example, David Lazer, Gary King, and colleagues illustrated general challenges of working with these data by focusing on Google Flu Trends. So skeptics are in good company. Still, where there is a business case, there also is hope.

Building on earlier work, in our new paper we focused on checking the central claim underlying the arguments of proponents of using Twitter to predict elections, implicitly assuming that mentions of political actors somehow indicate political support. On the face of it, this argument seems highly unlikely but a series of apparently positive findings give the impression that there might be something to it. If we look closely at the data, though, we find that Twitter mentions much more likely indicate public attention towards politics, a concept that sometimes–but far from always–might be correlated with political support. As we state in the article:

“Our evidence raises doubts with regard to the validity of Twitter-based metrics as indicator for political support. In addition, trends in the daily mention counts of political parties showed no systematic link with trends in opinion polls. Instead, the dynamics in the daily mention counts of parties appear to correspond with media events, media coverage of politics, and controversies. It, therefore, appears far more likely that Twitter-based metrics measure public attention toward politics than political support. Sometimes attention might be a covariate of support, but this relationship is far from stable. Our analysis underscores the importance of approaching the use of digital trace data in the measurement and analysis of political and social phenomena cautiously and emphasizes the importance of using established standards of social science methodology in indicator validation to avoid premature conclusions.”

Thus, studies identifying Twitter to be a predictor of electoral results might in fact have identified cases in which public attention, manifested in Twitter messages, was correlated with public support. But, as the many negative findings by us and other researchers indicate, this relationship is far from universal. To use Twitter as indicator of electoral fortunes might thus be simply betting on the stability of the link between attention and support in any given election. A stability which various politicians at the center of controversy and scandal might doubt.

To be sure, this is not to say that Twitter-data do not hold significant potential for public opinion research. As we have stated elsewhere, Twitter messages mirror parts of social and political life mediated through the interests, attention, and motivations of Twitter users. There is a host of topics in which Twitter-data might be productively used:

Likely candidates for political phenomena to create digital traces are political media events, intense media coverage of politics, or public controversies. Accordingly, future research may focus on using Twitter data to analyze which kind of political information attracts Twitter users’ attention and is distributed online. This gives rise to important questions concerning the sources, that is, the media, political elites, or social networks, and conditions successful in getting Twitter users to pay attention to political information. Thus, Twitter has the potential to become a source of insight into conditions and dynamics of attention toward politics.

Unfortunately, much of recent research attention has been focused on trying to make Twitter-data fit other measurements of political and social life. While the hope for finding cheap and readily available proxies for expensive and slow survey-based measures of public opinion is understandable, Twitter might be an unlikely candidate to provide this proxy. So instead of overextending our expectations scholars should probably leave predictions to consultants and instead:

“[…] should acknowledge the conditional nature of findings more freely and be more careful in considering and analyzing the consequences of potentially varying data-generating mechanisms. This might lead Twitter-based research to free itself from inflated early expectations to find proxies of public opinion in Twitter-data and instead focus on the potential of digital trace data in yielding insights into public attention toward political information. Digital trace data may thus provide valuable information for public opinion research, though on different phenomena than those on which prior research focused.”

Andreas Jungherr, Harald Schoen, Oliver Posegga, and Pascal Jürgens. 2016. Digital Trace Data in the Study of Public Opinion: An Indicator of Attention Toward Politics Rather Than Political Support. Social Science Computer Review. (Online First). doi: 10.1177/0894439316631043 [Online Appendix]

This paper’s findings have be covered by The Hill, The Guardian, The Washington Post, Newsweek, Reuters, Ars Technica UK, Fortune, Politico EU, The World Economic Forum, and others.

Syllabus: Explaining Political Behavior and Attitudes through Political Psychology

For this spring semester, I designed a new course introducing students to the use of concepts from political psychology and political sociology in explaining political behavior and attitudes. The course is a mix of theory and practical exercises.

In the first part of the course, students are introduced to a selection of central concepts and their operationalization in publicly available data sets from the USA and Germany like the American National Election Studies (ANES) and German Longitudinal Election Studies (GLES). In the second part, students are asked to use these concepts and datasets in individual research projects. The aim of the course is to introduce students to translating theoretical concepts into operationalizations and examine them through simple data analyses. Here is the syllabus.

Readings:

Background Readings:
Basbøll, T. Research as a Second Language.
Berinsky, A. J. (Ed.). 2016. New Directions in Public Opinion. 2nd ed. New York, NY: Routledge.
Clawson, R. A. & Z. M. Oxley. 2013. Public Opinion: Democratic Ideals, Democratic Practice. 2nd ed. Los Angeles, CA: Sage.
Cottam, M. K., E. Mastors, T. Preston, & B. Dietz. 2015. Introduction to Political Psychology. 3rd ed. New York, NY: Routledge.
Kabacoff, R. I. R in Action: Data Analysis and Graphics with R. 2nd ed. Shelter Island, NY:
Manning.
Kaplan, D. T. (2012). Statistical Modeling: A Fresh Approach. 2nd ed. Project MOSAIC.

Ideology
Federico, C. M. 2016. The Structure, Foundations, and Expression of Ideology. In: A. J. Berinsky (Ed.). New Directions in Public Opinion. 2nd ed. New York, NY: Routledge (pp. 81-103).
Federico, C. M. & C. V. Hunt. 2013. Political Information, Political Involvement, and Reliance on Ideology in Political Evaluation. Political Behavior 35(1): 89-112.
Neundorf, A. 2011. Die Links-Rechts-Dimension auf dem Prüfstand: Ideologisches Wählen in Ost- und Westdeutschland 1990-2008. In: R. Schmitt-Beck (Ed.). Politische Vierteljahresschrift Sonderheft 45: Wählen in Deutschland. Baden-Baden: Nomos (S. 234-257).

Race
Czaja, E., J. Junn, & T. Mendelberg. 2016. Race, Ethnicity, and the Group Bases of Public Opinion. In: A. J. Berinsky (Ed.). New Directions in Public Opinion. 2nd ed. New York, NY: Routledge (pp. 104-123).
Lee, T. 2008. Race, Immigration, and the Identity-to-Politics Link. Annual Review of Political
Science 11: 457-478.
Wüst, A. M. 2011. Dauerhaft oder temporär? Zur Bedeutung des Migrationshintergrunds für Wahlbeteiligung und Parteiwahl bei der Bundestagswahl 2009. In: R. Schmitt-Beck (Ed.). Politische Vierteljahresschrift Sonderheft 45: Wählen in Deutschland. Baden-Baden: Nomos (S. 164-185).

Gender
Burns, N., A. E. Jardina, D. Kinder, & M. E. Reynolds. 2016. The Politics of Gender. In: A. J. Berinsky (Ed.). New Directions in Public Opinion. 2nd ed. New York, NY: Routledge (pp. 124-145).
Box-Steffensmeier, J. M., S. de Boeff, & T. M. Lin. 2004. The Dynamics of the Partisan Gender Gap. American Political Science Review 98(3): 515-528.
Giger, N. & S. Huber. 2015. Der Einfluss des Geschlechts auf Kandidatenbeurteilungen: Eine experimentelle Studie zu Kontexteffekten und individuellen Faktoren in Deutschland. In: T. Faas, C. Frank, & H. Schoen (Eds.). Politische Vierteljahresschrift Sonderheft 50: Politische Psychologie. Baden-Baden: Nomos (S. 333-359).

Political Partisanship
Hetherington, M. 2016. Partisanship and Polarization in Contemporary Politics. In: A. J. Berinsky (Ed.). New Directions in Public Opinion. 2nd ed. New York, NY: Routledge (pp. 146-164).
Carsey, T. M. & G. C. Layman. 2006. Changing Sides or Changing Minds? Party Identification and Policy Preferences in the American Electorate. American Journal of Political Science 50(2): 464-477.
Neundorf, A., D. Stegmueller, & T. J. Scotto. 2011. The Individual-Level Dynamics of Bounded Partisanship. Public Opinion Quarterly 75(3): 458-482.

Personality
Mondak, J. J. & M. V. Hibbing. 2016. Personality and Public Opinion. In: A. J. Berinsky (Ed.). New
Directions in Public Opinion. 2nd ed. New York, NY: Routledge (pp. 165-185).
Gerber, A. S., G. A. Huber, D. Doherty, C. M. Dowling, C. Raso, & S. E. Ha. 2011. Personality Traits and Participation in Political Processes. The Journal of Politics 73(3): 692-706.
Schoen, H. & M. Steinbrecher. 2013. Beyond Total Effects: Exploring the Interplay of Personality and Attitudes in Affecting Turnout in the 2009 German Federal Election. Political Psychology 34(4): 533-552.

Religion
Campbell, D. E., G. C. Layman, & J. C. Green. 2016. A Jump to the Right, A Step to the Left: Religion and Public Opinion. In: A. J. Berinsky (Ed.). New Directions in Public Opinion. 2nd ed. New York, NY: Routledge (pp. 232-257).
Barker, D. C. & C. J. Charman. 2000. The Spirit of Capitalism? Religious Doctrine, Values, and Economic Attitude Constructs. Political Behavior 22(1): 1-27.
Roßteutscher, S. 2011. Die konfessionell-religiöse Konfliktlinie zwischen Säkularisierung und Mobilisierung. In: R. Schmitt-Beck (Ed.). Politische Vierteljahresschrift Sonderheft 45: Wählen in Deutschland. Baden-Baden: Nomos (S. 118-140).

Group Identity
Abdelal, R., Y. M. Herrera, A. I. Johnston, & R. McDemott. 2009. Identity as a Variable. In: R. Abdelal, Y. M. Herrera, A. I. Johnston, & R. McDemott (Eds.). Measuring Identity: A Guide for Social Scientists. Cambridge, MA: Cambridge University Press (17-32).
Sylvan, D. A. & A. K. Metskas. 2009. Trade-offs in Measuring Identities: A Comparison of Five Approaches. In: R. Abdelal, Y. M. Herrera, A. I. Johnston, & R. McDemott (Eds.). Measuring Identity: A Guide for Social Scientists. Cambridge, MA: Cambridge University Press (72-109).
McClain, P. D., J. D. Johnson Carew, E. Walton, Jr., & C. S. Watts. 2009. Group Membership, Group Identity, and Group Consciousness: Measures of Racial Identity in American Politics? Annual Review of Political Science 12: 471-485.