Scoring local news with NLP + AI · Federico D'Ignazio

An automated first pass for a local-media observatory: it scrapes a region's papers, scores each one with sentiment and bias heuristics plus an LLM summary, and hands the volunteer team a reviewable shortlist. The editorial call stays human.

Problem

Local journalism shapes how a community sees itself. But reading a region’s papers closely enough to notice how a topic gets covered means working through hundreds of articles by hand, and no small volunteer team can keep that up week after week.

What I built

An automated first pass. It reads the pile so a human team can spend its judgement where judgement is needed. The tool scrapes the region’s local papers and pulls the articles that mention the tracked terms. It runs sentiment analysis and a set of bias heuristics that flag potentially harmful or prejudiced phrasing. It has an LLM write a faithful summary of what each article says. Then it scores each one and rolls the results into per-outlet figures the team can follow over time.

the automated first pass

scrape the region's papers→ match tracked terms→ sentiment + bias heuristics→ LLM summary→ score, then roll up per outlet

Then a person makes the call. Fusing the classic NLP signals with the LLM summary into a single pass was the part that took care: the output has to be consistent enough to compare across outlets, while never standing in for the editor who decides what it means.

Where the tool sits

The observatory publishes its own method in three steps: Rilevazione, Revisione, Analisi (detection, review, analysis). My first pass lives in the detection step and preps the review; the analysis, and every judgement in it, stays with the human team.

The observatory's transparency panel, headed "è un lavoro di lettura critica" (a work of critical reading), with three cards: tracked sources stay visible, a human review on every published card, and evidence weighed before conclusions From oltrealtitolo.ch: tracked sources stay visible, every published card carries a human editorial pass, and the observatory’s line holds throughout. “La decisione editoriale resta umana”: the editorial decision stays human.

Where it stands

More than 2,000 articles are under review so far. That figure is my own count from the backend; the public site publishes no numbers. The results are being published soon. The observatory is deliberately running two workflows side by side: a fully human-led one and this NLP-assisted one, with a university collaboration planned, so the hard impact numbers will come once there’s something to compare.

Notes

Built as pro-bono engineering through CIPU for GMSI’s “Oltre al Titolo” media observatory (oltrealtitolo.ch). The observatory is explicit that this is critical reading, not a ranking of outlets or people, and I kept that principle in the design: every published card keeps its original source visible, and the machine only ever narrows the pile.