Honest Report Card — Numbers Measured in a Real Feed
The honest report card is the version of labeling performance measured not in a laboratory environment, but on a natural distribution real feed.
The measurement was taken on July 17, 2026, on 1110 titles that never went into training, with calibrated thresholds on the device. Values are given separately by language, because the model behaves differently in every language. P shows how many of the ones we labeled were correct, R shows how many of the ones that should have been labeled we caught. The weak rows are also staying in the table; there are no cut or rounded numbers.
| Task | P | R | F1 |
|---|---|---|---|
| Low Quality · TR | 0.99 | 0.61 | 0.75 |
| Low Quality · EN | 0.75 | 0.43 | 0.55 |
| Critical · TR | 0.90 | 0.52 | 0.66 |
| Critical · EN | 0.87 | 0.87 | 0.87 |
| Sentiment · Negative · TR | 0.76 | 0.80 | 0.78 |
| Sentiment · Negative · EN | 0.75 | 0.69 | 0.72 |
| Sentiment · Neutral · TR | 0.83 | 0.86 | 0.85 |
| Sentiment · Neutral · EN | 0.68 | 0.84 | 0.75 |
| Sentiment · Positive · TR | 0.67 | 0.48 | 0.56 |
| Sentiment · Positive · EN | 0.86 | 0.46 | 0.60 |
Example: In Low Quality TR, precision is 0.99, recall is 0.61. When it stamps the label, it almost never errs, but it catches only a portion of the low-quality items.
Go deeper
Even the scores of a test set that never went into training (held-out) are too optimistic: in the same task, Low Quality TR comes out as 0.95 there. The reason is that that pool is a laboratory environment enriched by rare class mining. Only the natural distribution report card above reflects the performance in a real feed.
Thresholds are chosen not to enlarge F1, but to maximize recall without falling below the precision floor. The floor is language-specific: 0.90 for low quality TR and EN, 0.90 for critical EN, 0.80 for critical TR. This is the reason why recall values appear low and it is a conscious choice — missing is preferred over stamping a wrong label.
Positive sentiment is the weakest link in every language: it is both rare and the “if in doubt, neutral” rule deliberately leaves it unprotected.