Moderating a feed without watching the people in it

Snakken's job is to show you what is happening on the handful of streets where you live. Doing that safely means moderation: keeping the neighbourhood feed clear of abuse, scams, and the genuinely illegal. The trouble is that the standard moderation playbook is built for platforms that watch their users closely — and watching users closely is the one thing our architecture is designed not to do.

This is us working through that contradiction in the open, with the mid-2026 state of the art on the table. It is a design exploration, not a deployment manual: lessons, not a map of our internals.

The state of the art, mid-2026

Modern content moderation has stopped being a single classifier. The well-run version is a tiered system: a cheap, fast model clears the obvious traffic, a large language model acts as a judge on the ambiguous middle, and humans review the genuinely hard cases. The whole design is governed by one trade-off — nuance against cost.

What changed is the middle tier. Frontier LLMs can read intent, sarcasm, and reclaimed slurs that keyword filters always mangled, and they enforce a policy written in plain language across dozens of languages without a separate model per language. A Google study this spring found LLMs now sit at every stage of abuse detection. The catch is over-refusal: a safety-tuned model used as a classifier tends to flag benign text that merely looks like the unsafe kind, so a system that leans on it too hard quietly censors the neighbourhood it was meant to protect.

Regulation has caught up too. Under the EU's Digital Services Act, every restriction we place on a piece of content owes the author a statement of reasons, and platforms now file standardised, machine-readable transparency reports — including the measured error rate of their automated moderation. The first harmonised batch landed in February 2026. Accuracy is no longer a private metric; it is a number you publish.

The hardest category is the one with the least room for error. Hash-matching against known abuse imagery (the PhotoDNA approach) is losing ground to AI-generated CSAM, which is novel every time and so matches nothing. On the regulatory side, the EU's "Chat Control" derogation that let big messengers scan private chats was rejected by Parliament in March 2026; its permanent successor is still in trilogue, aimed at a deal around July. The cryptographers' position has not moved: client-side scanning breaks end-to-end encryption no matter how it is dressed. The approach that has held up is Apple's — an on-device model that warns locally, with nothing uploaded and nothing reported.

The signals we don't have

Here is where most of that playbook stops being available to us. Large platforms moderate as well as they do because they watch: a behavioural profile per account, the full social graph, device fingerprints, and a location history going back years. Those signals are what catch the coordinated scam ring and the ban-evading repeat offender long before a human looks.

We threw all of them away on purpose. Our servers never receive your coordinates — the phone resolves its GPS fix into a coarse H3 cell and sends only the cell, so there is no location history to mine. There is no tracking, no behavioural model, no attention metric. Accounts are passkey-only, and the product is hyperlocal by construction, so the cross-platform graph that powers reputation scoring simply does not exist on our side.

That is a deliberate hole in our toolbox, and pretending otherwise would be the dishonest move. Automated moderation that depends on surveillance is automation we cannot run.

What a hyperlocal feed gives us instead

What we have instead is locality, and it turns out to carry real weight. On a global platform a harmful post is context-free; on Snakken the audience for any post is the same neighbours who can recognise it for what it is. The reporter, the harmed party, and the context that settles the call are all in the same few streets. Local knowledge is a moderation signal that scales down instead of up.

So the design space we are converging on leans on three things rather than on watching people. First, content and local reports do the heavy lifting — a report from someone who lives on the street is worth more than a behavioural hunch. Second, on-device classification fits us naturally, because the phone is already the thing doing the privacy-preserving work; an Apple-style local warning needs no server to see the content at all. Third, an LLM judge written against a plain-language policy can take the ambiguous middle, run with the over-refusal caveat firmly in mind and a human behind it.

And we treat the DSA as the floor, not the ceiling. A statement of reasons on every action, a real appeal, and published error rates are obligations we would want even if no regulation required them. Our one deliberate retention already follows that logic: reported content is held, not publicly visible, only until the report is resolved.

What we are not doing is dressing up abdication as empowerment. The industry's pivot toward community-notes-style moderation is partly a genuine good and partly a way to shed responsibility onto unpaid users, and it rests on optimistic assumptions about how diverse and diligent a crowd will be. The neighbours help; the accountability stays with us.

The other half: what the state can ask us to keep

Moderation is only half of the question. The other half is what a platform retains — because whatever you keep, someone can eventually compel. For a service built on data minimisation this is not a side issue; it is the same design surface seen from the other end. The less we hold, the more often the only honest answer to a data demand is "we never had it."

This is exactly the debate playing out in German law right now, as the country looks for what comes after blanket data retention (Vorratsdatenspeicherung). Two alternatives have framed it for years:

• Quick Freeze. Nobody stores traffic data in reserve. When investigators have a concrete suspicion of a serious crime, they order a provider to freeze whatever data still exists; a court order then releases it. The privacy gain is real — no mass, suspicionless storage — but it only works on data a provider happened to still hold, and critics warn a vague freeze order ("preserve a whole city's mobile data") can rebuild mass retention through the back door.
• The Login-Falle (login trap). When a crime is tied to a specific account, the platform is notified and captures the IP address the next time that account logs in, then hands it to investigators. No advance storage of anyone's data; collection is triggered by one suspected account actually being used. It only works for account-based cases, it depends on the platform cooperating, and it identifies a connection, not the individual behind a shared one.

In April 2026 the federal cabinet approved a draft that pairs a Quick Freeze procedure (§ 100g StPO) with a three-month obligation to store IP addresses (§ 176 TDDDG) — no content, no location, no movement profiles, but enough that privacy groups read the IP part as data retention by another name. It is a draft, not yet law, with its parliamentary fight still ahead.

Our stance follows from the architecture rather than from any one bill. We would rather operate in the freeze-and-login-trap world than the store-everything world, because targeted, court-ordered, after-the-suspicion access is the lawful-access design that fits a platform built to keep as little as possible. A login-trap-style request asks us for one fact about one suspected account at one moment. Standing logs ask us to surveil everyone forever on the off chance. The first is something a privacy-first platform can answer honestly; the second is the thing we built ourselves not to be able to do.

What this costs

Honesty requires the trade-off list, and this one is long.

Throwing away behavioural signals means our automation catches less, and later, than a surveillance-heavy competitor's. We lean harder on reports, which means harm has to be seen by a neighbour before the machinery starts — slower, and uneven across quieter cells. On-device models are bounded by what a phone can run and cannot be silently re-pointed at new targets, which is a safety property and a capability limit at once. The LLM judge over-refuses and costs real money per call. We are small and hyperlocal, so we cannot fund the around-the-clock trust-and-safety floor a giant can.

And the one that matters most to say plainly: privacy-first is not unaccountable. A login-trap-style order, lawfully issued, still ends with us identifying a connection. We keep little, we resist the suspicionless, and we publish what we do — but "we can't see your coordinates" was never a promise that a court can never reach a serious-crime suspect through us. Conflating the two would be the comfortable lie, and the whole point of this notebook is not to tell those.

Where this goes next

Three things will move before we have settled answers. The CSAR ("Chat Control 2.0") trilogue is aimed at a deal around July; the German IP-storage and Quick Freeze draft has its parliamentary fight ahead; and AI-generated abuse imagery keeps eroding the hash-matching that detection has relied on for a decade. We will write again when those land, and when our own design has more than principles to show. For now the principle is the deliverable: moderate on locality and content, not on surveillance, and keep so little that the honest answer to most demands is that we never had it.