On-Device Classifier

It is the built-in classical machine learning system of about 7 MB in size that measures the quality, criticality, and sentiment of the news on your device.

Instead of using massive models in the cloud, MagPunk runs small-sized (≈7 MB) classical machine learning algorithms integrated into your device. Trained with L2 normalization in a space consisting of letters and words with around 16,000 (8,000 + 8,000) dimensions, this lightweight model rapidly calculates the statistical tone of every news item that comes to you.

Example: Your phone being able to calculate the negative word density in the news within seconds even when you have no internet.

Go deeper

This model was trained on thousands of titles taken from the real feed (the training set in version v1.0.0 consists of 42,362 titles, and 85% of this, 36,002 data points, are used for training, while 15%, 6,360 data points, are used as the held-out test; seed=42 was used for deterministic splitting in the model). The operation of the model on the device is completely transparent: The model calculates word weights with the formulas TF = 1 + ln(occurrence) and IDF = ln((1+N)/(1+df)) + 1. It is trained with the L-BFGS method with a maximum of 2000 iterations. The decision mechanism turns into a probability by entering the sigmoid function over z = b + Σ(wᵢ·xᵢ).