Under the EU's Digital Services Act, platforms must declare for every reported moderation decision whether an automated process detected or removed the content.
That requirement created a regulatory expectation of measurable automation accuracy. It did not create a way to measure it. For text, a decade of shared benchmarks makes vendor claims checkable. For live video, there is no equivalent, which means every accuracy figure quoted in this category is self-reported and effectively unfalsifiable.
Fields move when benchmarks arrive. ImageNet did it for vision; GLUE and its successors did it for language. Live multimodal stream classification has no such artifact, and the absence is holding the subfield back more than any modelling limitation.
What follows is a specification sketch: what a credible benchmark here would have to contain, and why each property is non-negotiable.
Why the existing benchmarks do not transfer
The instinct is to assemble one from parts that already exist. It does not work, for three reasons.
Video classification datasets are built from complete clips with known boundaries. A moderation system never sees a complete clip; it sees a sliding window of an event still in progress.
Image safety datasets are frame-level and context-free. The same frame can be benign or a violation depending on the thirty seconds preceding it, and a frame-level label cannot encode that.
Audio datasets assume clean speech. Live sessions deliver overlapping talk, background noise, music, and two speakers on different latency paths.
Each existing resource measures something adjacent to the task. None measures the task.
The properties a real benchmark would need
1. A latency budget as a first-class axis
Accuracy reported without a time constraint is not a useful number here. A system that classifies correctly in forty seconds has failed at a task where the harm occurs in four.
The benchmark would therefore score models on an accuracy-at-latency curve rather than a single figure, the way speech recognition is scored against real-time factor. Submitting a slower, larger model would be permitted and would simply land further right on the curve.
2. Streaming evaluation, not clip evaluation
Models would receive the session as a stream, make decisions at fixed intervals, and be scored on when they acted as well as whether they acted.
This matters because it penalises the most common production shortcut: waiting for more evidence. A model that is always right eventually is worth less than one that is usually right quickly.
3. Cold-start pairs
Here is the property most likely to be omitted, and the one that separates easy cases from hard ones.
Moderation systems on established social graphs can lean on history — prior reports, account age, mutual connections, past behaviour. Those signals carry a surprising share of the load, and a benchmark built only on such sessions would flatter every model tested.
A random video chat platform has none of them. Two strangers are paired, the session starts cold, and the classifier has nothing but the stream itself. Any benchmark that aims to measure the model rather than the metadata has to include a substantial share of these cases, and report them as a separate slice.
4. Asymmetric cost weighting
A single F1 score hides the thing practitioners actually care about.
False negatives and false positives have different costs in this domain, and the ratio is not one that a benchmark designer should pick arbitrarily. The workable approach is to publish results across a range of cost ratios, letting operators read the curve at whatever weighting their jurisdiction and risk posture imply.
5. Multimodal fusion under misalignment
Audio and video arrive on different clocks. Real sessions include packet loss, resolution drops and partial frames.
A benchmark that supplies perfectly synchronised streams measures a problem nobody has. Degradation should be in the test set by design, not treated as an edge case.
The data problem nobody wants to discuss
All of this runs into an obstacle that is ethical before it is technical.
A representative benchmark would require recorded sessions involving real people, including sessions containing exactly the material the benchmark exists to detect. Collecting, storing and distributing that corpus is legally fraught and, for some categories, impossible to do responsibly at all.
Three partial routes exist, and none is clean.
Synthetic and acted sessions can cover a surprising amount of ground, particularly for the mechanics — detecting that two streams desynchronised, or that a frame was obscured. They are weak on the subtleties that make real cases hard.
Federated evaluation keeps data with each platform and ships the model to it, returning only aggregate scores. This preserves privacy at the cost of reproducibility, which is a substantial thing to give up in a benchmark.
Consented research panels — participants who agree in advance to have sessions recorded for safety research — produce the highest-quality data and the smallest volume, and they skew toward people willing to be recorded.
A serious effort would probably combine all three and be explicit about which slice each result came from.
Who would actually build it
Not a single platform, because a benchmark authored by one operator will encode that operator's product assumptions.
The plausible configurations are a consortium of platforms with an independent academic host, or a regulator commissioning it as part of the transparency regime that created the demand for the numbers in the first place. The second has more authority and moves more slowly.
Either way, the field is currently in an unusual position: it is subject to accuracy expectations in law while lacking any shared instrument for measuring accuracy. That gap is not stable, and whoever closes it will set the terms for everyone else.