Finding local topics without sending posts to US AI services

When there is a public-viewing crowd three streets over, or the river is rising over the embankment, the feed should be able to surface that by itself — not because we added a "flooding" category last week, but because it is what the neighbourhood is suddenly posting about. A feed that can only show the topics we thought of in advance is not really local; it is a menu.

So we wanted topics to emerge: to appear when enough people nearby are posting about the same thing, and to fade when they stop. This is the story of how we are building that on our own hardware in the EU, how we keep a topic from turning into a spotlight on one person, and the tuning problem that a few dozen short German posts handed us along the way. As with everything in this notebook, it is a design worked through in the open — lessons, not a map of our internals.

The constraint that shaped everything

Every design here starts with the thing we will not do. This time it was one sentence: no user's post ever leaves our own infrastructure to be read by a model we do not run. That rules out the obvious, cheap path — handing the text to a US-hosted AI API and getting excellent vectors back for a fraction of a cent. The moment a neighbour's post about a custody dispute or a protest is shipped abroad to be analysed, the promise that Snakken keeps European data in Europe has a hole in it, and "we only sent it for embeddings" is not a hole anyone should have to trust.

The models that turn a post into something we can compare run on machines we own, in the EU. That constraint narrows the field more than you might expect, because the licence matters as much as the benchmark. The open model we rated highest on quality turned out to ship under a non-commercial licence — unusable for a paid product without special terms — so we took a slightly less celebrated one, published under a permissive licence we could actually host and run commercially. The lesson we keep from that: once you self-host, "best on the leaderboard" and "legal for us to run" are different questions, and the second one wins.

Not everything needs the heavy machinery

Some topics announce themselves in the words alone, and it would be a mistake to reach for embeddings to catch those. When a word suddenly runs hot in an area — "Hochwasser" climbing far above its own normally quiet baseline — you can find it by counting, comparing the word frequencies in a short recent window against a longer one with a log-odds measure. "und" climbs too, in raw terms, but it never really leaves its baseline, so it is not a topic. This part needs no understanding of language at all. It is arithmetic, and cheap enough to run constantly.

What counting cannot do is put together posts that mean the same thing in different words. "The river is over the wall" and "Sandsäcke am Deich" share nothing lexically and belong to one topic, and only a model that has read enough language knows that. That is the job the embeddings do, and it is also the operationally heavier one: vectors to compute, clusters to form, a model to keep serving. We were deliberate about where each approach earns its place: lean on the cheap statistical signal for what it can already catch, and spend the semantic machinery only on the meaning-level grouping that genuinely needs it. Reaching for the heavy path everywhere, including where a word-count would have done, is how you end up paying to run infrastructure that something far simpler could have replaced.

One service, one seam

The tempting shape for the semantic part is a small Python service: the machine-learning ecosystem lives there, BERTopic and its neighbours are a pip install away, and it feels tidy to give topic modelling its own box. We did not do that, and the reasoning is a rule we keep returning to: add a service boundary only where the runtime genuinely differs, not where the subject matter feels different.

The only part of this that truly needs a different runtime is the model, and that already runs on our own hardware behind a small serving layer. Everything around it — pulling the recent posts for an area, asking for their embeddings, comparing vectors, deciding what counts as a cluster — is ordinary application work our existing worker already does well. Cosine similarity between two vectors is a dot product; it does not need a data-science service to compute. The embeddings come from the model we already run, and the clustering stays in code we already maintain and watch. That is one less service to deploy, secure, and get paged for.

A topic can point at a person

A topic is a small act of aggregation, and aggregation is where hyperlocal privacy turns subtle. On a city-wide platform, "people are posting about X" reveals nothing about any one person. On a few streets, a topic assembled from two posts by one worried resident is not a topic — it is that resident, re-broadcast with a label on them. The smaller the area, the more a "topic" can become a spotlight.

A candidate topic does not surface until it clears a k-anonymity gate: it has to be carried by at least a minimum number of posts from at least a minimum number of distinct authors before it can appear as a chip in the feed. One loud voice is not a neighbourhood trend. Below the gate, the topic simply does not exist as far as the feed is concerned.

The embeddings themselves get the same treatment as the coordinates they descend from. A post's vector is owner-only derived data — it belongs to the author's own record, fenced off with row-level security, and never pooled into a shape that would let one account's writing be matched or reconstructed from another's. It is a means of grouping posts for a moment, not a durable profile of a person. This is the same instinct as resolving the H3 cell on the phone: derive only what the feature needs, keep it next to its owner, and make the privacy-preserving version the only version the system can express.

Ship, observe, tune

None of this survived contact with real posts unchanged, which is the point of running it against a test batch before trusting it. The first time we put the semantic layer over a set of genuine short German posts, they refused to cluster. Posts we could see with our own eyes were about the same street festival came back as unrelated.

The reason was the similarity threshold. Posts that were plainly related were scoring just under the cutoff we had set for what counts as "the same topic" — a value we had taken, more or less unexamined, from benchmarks built on long, well-formed English documents. Real neighbourhood posts are short and mixed-language, stuffed with abbreviations and place names; they never build up the overlap that a tidy paragraph of prose does. The threshold was not wrong in general. It was wrong for our text.

The fix taught us more than the bug did. We had reached for the obvious lever — change the number, redeploy, watch — and realised that loop was far too slow for a value we would clearly be adjusting for weeks. The threshold is no longer a constant baked into the build; it is a runtime parameter an operator can change from the admin panel and see take effect on the next pass, with no deploy and no restart. A value you tune by observation belongs somewhere you can turn it while you observe.

What this costs

Honesty requires the trade-off list. Running our own models in the EU means we carry an operational burden a hosted API would have carried for us: a model to keep serving, a machine with enough memory to hold it, throughput and latency that are now ours to own. Refusing the US endpoints also means refusing, for now, whatever quality edge the largest proprietary models might offer — we take a good open model we can host over a better one we cannot.

The privacy gate that stops a topic from becoming a spotlight also makes Snakken slower to show a genuinely emerging one than a platform that surfaces anything the moment two people mention it. In a quiet cell, a real early signal can sit under the gate and never appear. And a threshold an operator adjusts by hand leans on judgement: we made tuning easy, not automatic, and a lever that is quick to change is also quick to get wrong.

Where this goes next

The open question is the tuning: whether a similarity threshold set by hand should, in time, become one the data sets for itself, and how the cheap signal and the semantic one should be reconciled when they disagree about what counts as a topic. The parts we would keep whatever model ends up underneath are the settled ones — the k-anonymity gate and the owner-only vectors. When there is more to show than a working design and an honest bug list, we will write again. For now the deliverable is the shape: topics that surface because a neighbourhood is talking, worked out on hardware we control, in a place where a post about your street never has to cross an ocean to be understood.