Text Analysis
Alice McGrath
March 6, 2025
Agenda
- Final project proposal questions
- Text analysis examples
- Cultural analytics tutorial
Project proposal questions
Text Analysis Methods
From the Google Books NGram
viewer
Natural Language Processing
- Bag-of-words: word counts, regardless of word order
- Language models: machine learning algorithms trained on language-specific datasets
Word counting
Methods
- N-gram frequency (unigrams, bigrams, trigrams)
- KWIC: Keywords in context/concordance
- Collocations: words that appear in close proximity
- Topic modeling: sets of words that appear together across multiple documents
- TF-IDF: term-frequency inverse-document-frequency
Examples
- Fan Engagement Meter - Peter Decherney, James Fiumara, Scott Enderle (Price Lab) - quantifying text reuse in fan fiction archives from popular film franchises
- On The Books - UNC - identifying Jim Crow laws that have not been repealed
Language models
Methods
- Named Entity Recognition/parts-of-speech tagging
- Word vectors/word embeddings
- Sentiment analysis*
- Text classification
- Text prediction/generation
Examples
- Sentiment analysis of Weimar film reviews, Campregher Paiva & Diecke
- Stanford Lit Lab: Domestic Spaces in Fiction
TF-IDF
TF-IDF with SciKit Learn, from Intro to Cultural Analytics in Python.
- Access the notebook from your Python Github Classroom assignment
- Merge pull request
- Download to your desktop or open in CodeSpaces
Other resources
Tools and resources
- Voyant-tools.org - no-code corpus exploration
- Constellate no-code tutorials and jupyter notebook tutorials for a range of methods