Matt Priselac

I.Where it started · NLP · Citation analysis · Web app

Quotation profiles

Which parts of a historical text does the scholarship on a topic actually lean on? I pulled every citation to Hume’s Treatise of Human Nature out of about a hundred papers on personal identity and pinned each one to the paragraph it points at. The result is a map of the topic drawn by the scholars themselves.

Read the dev diary ↓

The question

Hume’s Treatise is enormous: three books, each in three or four parts, each with seven to seventeen sections, plus an Appendix and an Abstract added more than a year after publication. A newcomer who wants to understand Hume on personal identity doesn’t know where to start. A specialist writing on the topic doesn’t always know who else has worked on the same passages.

My premise was simple. You can see where a philosophical topic lives in the Treatise by looking at where the experts cite it when they write about that topic. So I needed two things: a paragraph-level map of the Treatise, and a set of publications that someone other than me had already decided were “about” personal identity.

Defining the literature

I didn’t want to pick the papers myself. PhilPapers keeps expert-curated topic leaves, and its leaf on Hume on Personal Identity held 143 publications in June 2020, in several languages. I processed 101 of them, everything I could get in a usable digital form, some through interlibrary loan one chapter at a time. For books I used judgment: a book wholly about the topic went in whole, and otherwise only the relevant chapters did. The collection runs from 1955 to 2019, and about a quarter of it comes from Hume Studies.

The map of the text came from the Hume Texts Online edition, which numbers every paragraph. That gave 1,375 paragraphs in 93 sections, so a citation could be scored at either grain.

Pulling the citations out

Hume scholars cite the Treatise in at least four ways, and they changed their habits over the decades. The extraction works down a ladder, from the forms I’m sure of to the ones I’m not:

  1. Norton structural references like 1.4.6.4, in lists and ranges, sometimes with Roman numerals for the book or part.
  2. Explicit page references to the Selby-Bigge/Nidditch edition, like (SBN 252). Newer work often gives both forms, so a citation mustn’t be counted twice.
  3. Marked page references like (T 252) or (THN 252), common in Hume Studies.
  4. Bare page numbers like (252), which are everywhere in older work and might point at anything.

Strict search takes only the first three. Aggressive search adds bare page numbers with two or three digits. I dropped single digits after finding that (1), (2) and (3) made the Treatise’s short Introduction look as important as the section on personal identity itself. Page numbers become paragraphs through SBN-to-Norton concordance tables I built for the purpose.

Scoring and comparing

Every citation is worth one point, spread over the paragraphs it names. Each paper ends up with a score sheet over all 1,375 paragraphs, in eight versions (strict or aggressive, raw or relative, paragraph or section). Two papers are compared by the share of their citations that go to each location, so their distance runs from 0 (identical profiles) to 200 (no overlap). The literature as a whole is the sum of every paper’s sheet, so any one paper can be measured against the field.

What the map shows

Under strict search, about 2,760 citation points fall heavily on a few sections:

The Appendix’s 20th paragraph is the single most-cited paragraph in the whole Treatise. The aggressive search finds nearly twice as many citation points and keeps almost exactly the same order. For a reader, that’s a reading list. For a scholar, it means you can ask a different kind of question: who works on the same passages I do, and who leans on parts of the text that nobody else does?

The site

  • Literature as a wholeThe bar-chart map, plus a reading-list generator (“give me the top 15 paragraphs, strict search”).
  • Text locationsChoose a book, part, section or paragraph and see which publications cite it most.
  • Individual publicationsEvery paper ranked by how closely it matches the field, with its citations, a PhilPapers link and its nearest neighbours.
  • Project overviewA long methods page explaining the search, scoring and comparisons.

What I’d planned next

The next version would have let a scholar upload a draft and get back the published papers that lean on the same passages. After that, the plan was to repeat the whole process for other PhilPapers topics on Hume, and see which parts of the Treatise different topics share.

Development diary

Reconstructed from the git history, file dates and the running notes in the project’s notebooks. Dates before 27 May 2020 come from file dates, so they are approximate.

Building the corpus and the pipeline

October 2019 – mid-June 2020
Dev diary
Oct 2019 – Feb 2020

Building the pile

The first files in the folder are PDFs. One arrived in late October 2019, a dozen more on 7 November, and most of the rest on 6 February 2020. Every one is named by its PhilPapers ID, so from the start the PhilPapers leaf on Hume on Personal Identity was the definition of “the literature.” The spreadsheet that ties each file to its record keeps a “Downloaded?” column with a mixture of “yes”, “YES” and “Yes”, which says something about how it was kept.

Dev diary
13 April 2020

A machine-readable Treatise

I made the three files the rest of the project stands on: the full Treatise text with every paragraph numbered, and a pair of concordance tables between Selby-Bigge page numbers and Norton paragraph numbers in each direction. Without them a citation like (T 252) is just a number.

Experiment log
May 2020

From one file to a field

The oldest scripts took one paper, found its citations, and produced its distribution over the text. By 20 May there were plain-text versions of about a hundred papers and three notebooks working on what to do with them: one set up the comparison, papers as columns and paragraphs as rows; one settled how many neighbours to show (I can include 5 on either side as long as it’s within five percent); one added grades (90th percentile or better matches = A quality match). The initial commit came on 27 May 2020, with a refining-search branch opened within minutes. Everything after that happened there.

Method
Late May – mid-June 2020

Making the search smarter

The longest notebook in the folder reads like a lab diary. The key design decision is written down early: get nortons. if fewer than 5 nortons, get sbns if fewer than 5 sbsn, get strict parens if fewer than 5 strict parens, get loose parens. That ladder became the strict search, and the loosest step became aggressive. Other notes record the tedious work: cleaning up newly acquired PDFs, fixing the parser so it read the § sign, and compiling chapters of Penelhum’s Themes from Hume as they came in from interlibrary loan. By mid-June each paper produced eight score sheets.

A website in two weeks

18 June – 2 July 2020
Dev diary
18–19 June 2020

Tidying up for a website

Six commits in one day, all housekeeping before building anything new. The scattered scripts became one package with a Paper class, text processing, scoring, literature-level operations and comparisons. Old scripts and CSVs moved aside. The next morning’s commit says comp functions completed.

Launch
20–27 June 2020

The website in a week

Flask basics on 20 June. Individual publications and the list of all papers on the 21st, each paper’s citations on the 22nd, and on the 23rd the reverse lookup: pick a paragraph and see who cites it. A landing page on the 24th. At 10 pm on 27 June: fully functional first version of website, with the Literature page, its two Tableau dashboards, a reading-list generator and the long methods page.

Launch
28 June – 2 July 2020

Into the cloud

Next the site had to stop reading files from my laptop. A project-management notebook writes out the whole pipeline as a checklist and then works through Google Cloud Storage and Firestore one step at a time, ending with: the only thing i’ll need to do now is to replace the local file functionality with cloud file functionality! The commits follow it closely, to the first working deploy on 1 July and a last fix on 2 July 2020.

Dev diary
After July 2020

Where it stopped

The git history ends there. The methods page promised draft uploads and topics beyond personal identity; neither was built.

Threads through the diary

The literature wasn’t mine to define.I used an expert-curated PhilPapers leaf instead of my own reading list, so the map reflects the field rather than my view of it.

Precision against recall, made visible.The site never chose between strict and aggressive search. It showed both, side by side, and explained the trade-off.

A philosopher’s problem first.The web stack took a week. The hard part was knowing how Hume scholars have cited the Treatise since the 1950s, well enough to write a parser for it.