Precision, Recall and Threshold (P / R / F1)
They are the metrics that measure the success of a detection system; they express the balance between how few errors are made (precision) and how many of the targets can be found (recall).
Precision is the rate of the system being right when it says ‘this is so’. Recall is how many of the ones that are truly so it can find. The threshold value is the balance point between these two. While fewer but more accurate results are obtained when the threshold is raised, mistakes begin to mix in when the threshold is lowered.
Example: In a spam filter, if precision is high, real emails never fall into spam. If recall is high, no spam escapes.
Go deeper
In data analysis, the threshold value is never a fixed number (for example, z ≥ 0.5, which is merely an example); it is selected from the data according to the structure of the problem. By aiming for a high precision floor (e.g. 0.90 or 0.80) and automatically selecting the threshold, the system’s margin of error can be minimized, but in this case, the recall rate drops. The F1 score is the harmonic mean of these two values and shows the overall performance of the system. For instance, if P: 0.99, R: 0.61, and F1: 0.75 is achieved on low quality Turkish items measured on a real feed with a natural distribution, this means the system very rarely errs but can catch only a portion of the low quality ones.