Building Apps & Bots for Social Science Research

Chris Bail
Duke University

Website: https://www.chrisbail.net
Twitter: https://www.twitter.com/chris_bail
Github: https://github.com/cbail

Part I: APPS

Weaknesses of Digital Trace Data:

 

Incomplete
Inaccessible
Non-Representative
Drifting
Algorithmic Confounding
Dirty
Sensitive

But What Are the Alternatives?

 

But What Are the Alternatives?

 

-Survey response rates are low

But What Are the Alternatives?

 

-Survey response rates are low

-Many of our most important questions require longitudinal/relational/qualitative data

But What Are the Alternatives?

 

-Survey response rates continue to drop

-Many of the most important questions require longitudinal/relational/qualitative data

-Digital trace data have a number of major advantages that conventional sources do not (big, always on, non-reactive)

Can We Have Our Cake and Eat It Too?

Apps Can Address Many Limitations

 

Incomplete
Inaccessible
Non-Representative
Drifting
Algorithmic Confounding
Dirty
Sensitive

Example: The Viralgorithm

Social Media Survey Apps (SMSAs)

 

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

b) collect supplemental information from such users (e.g. demographics) using more conventional survey methods;

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

b) collect supplemental information from such users (e.g. demographics) using more conventional survey methods;

c) offer something back to the user as an incentive to share their data (e.g. analysis or financial incentives)

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

b) collect supplemental information from such users (e.g. demographics) using more conventional survey methods;

c) offer something back to the user as an incentive to share their data (e.g. analysis or financial incentives)

See: Bail 2015

Workflow of a Social Media Survey App

Workflow of the Viralgorithm

Response Rate

 

  1. Approximately 40% of all organizations contacted installed the app (across two fields)

Minimal Evidence of Selection

Examples

 

Bail, Christopher A. 2016. “Combining Network Analysis and Natural Language Processing to Examine how Advocacy Organizations Stimulate Conversation on Social Media.” Proceedings of the National Academy of Sciences, 113:42 11823-11828

Bail, Christopher A. 2016. “Emotional Feedback and the Viral Spread of Social Media Messages about Autism Spectrum Disorders.” American Journal of Public Health 106(7): 1173-1180

Bail, Christopher A., Taylor W. Brown, and Marcus Mann. 2017. “Channeling Hearts and Minds: Cognitive- Emotional Currents and Public Deliberation on Social Media.” American Sociological Review, 82(6) 1188-1213

Challenges of Building Apps for Social Science Research

 

  1. Significant coding skills required (html, css, cloud-computing, reactive programming)

  2. Competitive environment for attention (apps are no longer “new”)

  3. Concerns about data sharing/privacy

  4. Compelling incentives are hard to identify- and particularly challenging for studies of sensitive topics. But financial incentives may be an important option going forward.

OK, but How Do I Build an App?

Shiny

Shiny is a (relatively) new tool that enables people to build, compile, and host interactive apps natively within RStudio

Example

Check out the R code here.

Anatomy of a Shiny App

 

ui

 

The “face” of the app. Determines what user will see (e.g. what types of visualizations, check boxes or word boxes, fonts, etc.) Can load fancy images, logos, etc. to improve the overall appeal of the app.

server

 

The “brains” of the app- runs the analysis you want to show the user, but can also store data generated by the user, or expose different users to different types of information (good for experimentation)

Shiny Templates

Check out the Shiny Gallery here.

Linking to Cloud Servers

 

-Check out the googledrive package for loading and storing data.

-High-volume app hosting is available via RStudio.

Google's Flutter SDK

Part 2: BOTS

Bots are bad, right?

Using Bots to Study Social Media & Polarization

Using Bots to Study Social Media & Polarization

Using Bots to Study Social Media & Polarization

Bail et al. 2018

Using Bots to Study Social Media & Polarization


Bail et al. 2018

How Do I Build a Bot?

A Very Simple Bot

for (i in 1:24){

  #Search for 50 recent tweets about computational social science
  css_tweets<-search_tweets("Computational Social Science", n=50, include_rts = FALSE)

  #Randomly pick one tweet which appears in the `text` variable of `css_tweets` 
  lucky_tweet<-sample(css_tweets$text, 1)
  post_tweet(lucky_tweet)

  #Pause for 1 hour
  Sys.sleep(3600)
}

Making a Bot Interactive

Making a Bot Interactive

Making a Bot Interactive

From Munger, 2017

Hosting a Bot

Hosting a Bot

 

Check out my tutorial on running RStudio in the cloud here.

Storing Data

 

Ethical Issues in App & Bot-Based Research

Ethical Issues in App & Bot-Based Research