Reference, Similar, Unrelated

It is the similarity grading showing how much two news texts overlap over their words and semantic roots.

The system matches the news with each other by taking their fingerprints. If similarity is ≥ 50% the content is considered reference, if between 30-49% similar, if < 30% unrelated. The reference label is not an accusation of copying or plagiarism; high-quality publications that use the same official agency text and process the topic with exactly the same words naturally receive high similarity too.

Example: All of them being linked to each other as references upon a statement made by the Ministry being published with the same words on ten different sites.

Go deeper

For the comparison operation to be done efficiently, the system uses an inverted index architecture (O(N²) → ~O(N·k)) and this process takes place depending on 4 mandatory conditions. In matchings, conjunctions shorter than three letters or very short words are eliminated; the length of the eliminated word in the fingerprint is 3, while the root length is 4. The comparison window is limited to ≤ 24 hours; meaning only events within one day are connected to each other. As per system rules, the reference threshold can never be lowered below the similar threshold. All these operations take place on the device’s own hardware.