bag-of-words
A text-analysis method that ignores word order and treats a passage as an unordered collection of words, scoring it by aggregating the values of its constituent words.
As defined in 181 | Peter Dodds on Quantifying the Shape of Stories
Where it comes up · 1
Most important claims · 4 of 4Search all claims for “bag-of-words” →
- Analyzing emotional arcs across ~20,000 books reveals six fundamental shapes that cover roughly 90-95% of stories: rags-to-riches (rise), tragedy (fall, e.g. Metamorphosis), man-in-a-hole (fall-rise), Icarus (rise-fall), Cinderella (rise-fall-rise), and Oedipus (fall-rise-fall), distinguished simply by their number and arrangement of maxima and minima.181 | Peter Dodds on Quantifying the Shape of Stories
- When estimating the happiness of a set of words, the measurement error is not really in the sampled text but entirely in the lexicon—the list of words for which you have scores—so changing which words you score, or removing some, is where the variation comes from; the error is in the instrument itself.181 | Peter Dodds on Quantifying the Shape of Stories
- A hedonometer applied to Twitter data over 13-14 years functions like a Dow Jones index of happiness, normally rebounding within days after celebrity deaths or terrorist attacks because a flood of other stories washes them out, but in 2020 the pandemic produced the first true 'collective trauma'—a months-long depression rather than a brief dip—with George Floyd's murder being the lowest point ever recorded and January 6th the third lowest.181 | Peter Dodds on Quantifying the Shape of Stories
- The emotional arc shapes found in fiction are not artifacts of the method: if you randomly shuffle the text of a story, it does not produce these shapes, providing a control that confirms the arcs reflect real structure rather than the measurement procedure.181 | Peter Dodds on Quantifying the Shape of Stories