I’ve roasted coffee at home for years, and buying green coffee has always meant a dozen browser tabs. Every seller describes their coffees differently. One calls a region by the province, another by the washing station. One lists six tasting notes, another writes a paragraph. Some list the varietal, the altitude and the process; some list a country and a price.
Homegreens is my attempt to put all of that in one place and make it comparable. Every week a pipeline visits the US green-coffee sellers I follow, finds each coffee’s page, and keeps just the part of the page that describes the coffee. A language model reads that text and pulls out a structured record: origin down to region and farm, process, varietals, altitude, tasting notes and price. A second round of cleaning makes those records agree with each other. Spellings of regions are merged, varietals are canonicalised (SL-28 and SL 28 are the same plant; Pink Bourbon and Bourbon are not), and tasting notes are placed on the SCA flavor wheel, with notes about body, acidity and sweetness kept apart from flavor.
What’s on the site
DashboardWhat’s arrived recently, top origins, and which origins are in or out of their fresh season.
All Green CoffeesOne sortable, filterable table of every coffee, current and sold out.
Individual coffeesThe seller’s raw description next to the standardized one, a flavor wheel drawn from its notes, and the most similar coffees from other sellers.
Flavor AnalyticsWhich flavors are genuinely distinctive of a country, region or seller; what drives price; how origin, process and varietal relate; and how quickly coffees sell through.
AboutHow the data is collected and processed, written for roasters rather than engineers.
Why it’s here
This is the project where I learned to build software that leans on a model’s judgment, and then learned to check that judgment. Most of the interesting work wasn’t scraping. It was deciding what counts as “the same” region, flavor or varietal, and building tests to find out whether a model was classifying a tasting note the way a careful person would. That question, how far a model’s verdicts can be trusted and how you’d find out, is the one my research now asks directly.
“Haiku will too often key on just the first word: it categorizes green apple as a green/raw flavor.”My notes, 2024
Development diary
From a course project in 2023 to a weekly pipeline and a public site. Drawn from the git history, file dates and my own notes.
Before the site
2023 – spring 2024
Dev diary Summer 2023
Where it started: scraping Coffee Review
The project’s seed is a data-science course project. In July 2023 I proposed scraping all 7,000+ reviews on coffeereview.com, to see “what relationships can be found between price, coffee origin, and different ratings” and whether certain flavor notes go with higher scores. The proposal already names the end goal: combine review data with “what coffees I have available for purchase in order to make purchasing decisions for myself.” The reviews I collected that summer later became the training data for the site’s first prediction model.
Dev diary Dec 2023 – Mar 2024
One notebook per seller
Before writing any app I opened a Jupyter notebook for each seller and studied how its site was built: Coffee Bean Corral in December, then Hacea, Prime Green, Showroom, Mill City, Copan and Burman in early February, and a run of ten more on 19–20 February. A note from 19 February lists the “target unifying columns” every seller would need to map onto. By March I was testing whether an LLM could read a pruned copy of a product page and fill in those columns.
Dev diary Mar – Apr 2024
A scraper for every seller
In mid-March I wrote a small app per seller (fifteen of them dated 14–16 March), then folded them into one “full app” driven by a dictionary of per-seller functions: how to list coffee links, how to turn a URL into a name, how to trim a page to the part that describes the coffee. My notes from that month tick off each seller (“Coffee Project: all good to go. Added a filter function to get rid of blends”) and record the first real bug: Mill City’s links changed on every visit, so I switched from comparing links to comparing names. SWITCHING TO NAME BASED DOES THE TRICK. The first full runs are logged on 3, 6 and 12 April 2024, and the first LLM extraction code is dated 13 April.
Dev diary Apr – Jun 2024
Making the data agree with itself
With raw records flowing, the work became cleaning. I built separate LLM refining steps for location, altitude, process and flavor, each with a retry loop. Two lessons from my notes: Sonnet beat Haiku at classifying flavors because Haiku “will too often key on just the first word (i.e. it categorizes green apple as a green/raw flavor rather than a green [apple])”, and long retry loops should escalate to the stronger model rather than start over. I adapted the SCAA flavor wheel into a three-level taxonomy (family, genus, species) that a model could navigate step by step.
A public site
Summer 2024 – 2025
Launch Jun – Jul 2024
Launch
In late June I built the Streamlit front end and trained the first fun-only model: a logistic regression over 7,700+ Coffee Review assessments, predicting which of seven score buckets a coffee falls into from its origin and its tasting notes. The About page is blunt about what it isn’t: Do not use these predictions as a purchasing guide. The site’s first commit is 2 July 2024, and by 15 July the database had moved to AWS with read-only credentials. Royal Coffee Crown Jewels, Burman and Bodhi Leaf were added that month.
Dev diary Aug – Sep 2024
The data pipeline gets its own repo
On 8 August 2024 the pipeline got its own repository, separate from the website. The first weeks of commits are about duplicates and expired listings: “updated the expired duplicate removal”, “running several update cycles for fixing rows”, “troubleshooting location fixing problems”. From September the log settles into a rhythm of dated update runs (“sep 9 update run”, “sept 14 update run”).
Experiment log Oct 2024
Similar coffees, and a first evaluation harness
On 1 October the site gained “similar coffees”: a vector search over flavor and origin, shipped after a day of fighting dependency versions on Streamlit Cloud. The same month I tried running the flavor classifier on local models through Ollama and, more importantly, wrote my first evaluation plan for it: compare prompt-and-model combinations by the distribution of their classifications, score disagreements by distance on the wheel (3 for a different family, 2 for genus, 1 for species), and rank tasting notes by how hard they are to classify. This is the first place the project starts asking how good the model’s judgments are rather than whether the code runs.
Dev diary Feb – Apr 2025
Maintenance runs
Mostly scheduled data runs and seller fixes (Royal’s site changed in February). An April fix removed a dependency on a beta API feature.
Dev diary June 2025
From CSVs to a real database
The biggest rebuild. On 2 June 2025 the per-seller CSV files were replaced by a single SQLite database, followed within two weeks by database ingestion and LLM cleaning, a proper scraping package with logs and alerts, flavor embeddings for similarity, and a nightly sync to a cloud database on Supabase (16 June). The website moved to Supabase on the same day. A dated alerts report now accompanies every run.
Dev diary July 2025
Analytics and sold-out coffees
The site’s first analytics page shipped, along with prices and the ability to browse sold-out coffees.
Checking the model’s judgment
2026
Dev diary Jan – Feb 2026
Building with Claude Code
From January 2026 the commits are co-authored with Claude. The first such change cut LLM calls by about half by auto-assigning any tasting note whose classification had been at least 65% consistent historically (446 precomputed mappings). In February, two sellers had moved to Shopify and needed new scrapers, two new sellers were added (Prodigal and Dean’s Beans), and the site got a full analytics expansion: cross-feature associations, price interactions, turnover and flavor co-occurrence.
Dev diary July 2026
Sweet Maria’s moves
Sweet Maria’s moved to a new Shopify storefront and its scraper was rewritten.
Experiment log Sep – Oct 2026
Measuring the classifier, then replacing it
The most recent chapter is the one closest to my research. I built a 200-note gold set with a labeling guide, split into dev and test, and compared three ways of classifying tasting notes: the existing multi-turn Haiku tool loop, a single-call Haiku baseline, and TypeSafe’s jev classifier. On the 120-note test split, the existing pipeline put flavor-wheel labels on 94% of notes that contain no flavor at all (words about body or acidity); both alternatives brought that near zero. Species-level F1 rose from about 60 to about 89–91. jev was the most precise (97% species precision) and very repeatable (99% agreement across runs on flavors).
On 1 October 2026 the cleaning step was rebuilt around jev. Flavors follow the SCA wheel, while body, acidity, sweetness, finish and quality words go to a separate field modelled on the SCA cupping form instead of being forced onto the wheel. The similarity embeddings were rebuilt too: the old vector was only about 4% flavor, so v2.0 weights each block explicitly. I checked the result with a blind A/B test of old and new “similar coffees” lists and a tasting-note viewer for spot checks.
Two days later came a round of data repairs: Royal and Coffee Bean Corral moved to Shopify, expiry dates had silently stopped syncing to the cloud since June 2025, and Mill City’s links were once again changing on every visit, two and a half years after the first time. The flavor analytics were rebuilt on per-coffee data with corrected statistics (one-sided Fisher tests, Benjamini–Hochberg correction, and a rule that a flavor can’t count as typical of a country if one seller supplies most of its examples).