Intro to Text Analysis

Chris Bail
Duke University

WWI

Harold Laswell

Harold Laswell (1938)

 

“We may classify references into categories according to the understanding which prevails among those who are accustomed to the symbols. References used in interviews may be quantified by counting the number of references which fall into each category during a selected period of time (or per thousand words uttered).”

Ahead of his time?

Timeline of Quantitative Text Analysis

 

Time Activity
1934 Laswell Produces first Key-Word Count
1934 Vygotsky Produces first Quantitative Narrative Analaysis
1950 Gottschalk Uses Content Analysis to Track Freudian Themes
1950 Turing Applies AI to text
1952 Bereleson Publishes First Textbook on Content Analysis
1954 First Automatic Translation of Text (Georgetown Experiment)
1963 Msteller and Wallace analyze Federalist Papers
1965 Tomashevsky Further Formalizes Quantitative Narrative Analysis

Timeline of Quantitative Text Analysis

 

Time Activity
1966 Stone and Bales use mainframe computer to measure psychometric properties of text at RAND
1980 Decline of Chomskyean Formalism/NLP is Born
1980 Machine Learning is Applied to NLP
1981 Weintraub counts parts of speech
1985 Schrodt Introduces Automated Event Coding
1986 Pennebaker develops LIWC
1988 First Latent Semantic Analysis Patent
1989 Franzosi brings Quantitative Narrative Analysis to Social Science

Timeline of Quantitative Text Analysis

 

Time Activity
1998 Mohr's Quantitative Analysis of Culture
1999 Bearman et al. apply Network Methods to Narratives
2001 Blei et al. develop LDA
2002 MALLET created
2005 Quinn et al. use analyze political speeches using topic models
2009 New Directions in Text Conference
2010 King/Hopkins Bring Supervised Learning into mainstream
2010 Tools for Text Workshop at Washington

An Interdisciplinary Field!

 

Opening Black Boxes

 

From Words to Numbers

 

Other Great Introductions

 

Justin Grimmer & Brandon Stewart. Text as Data: The Promises and Pitfalls of Automated Content Analysis, Political Analysis.

James Evans & Pedro Aceves. Machine Translation: Mining Text for Social Theory. Annual Review of Sociology.