Model Monitoring

2 articles

Artificial Intelligence 07 Sep 2026 13 min read

Handle Label Shift in Deployed Classifiers

A classifier can keep seeing familiar inputs and still make worse decisions after deployment. One reason is that the frequency of the classes has changed. Imagine a model trained to classify support tickets as billing, account, or technical. During training, billing tickets made up 20% of examples. After a pricing migration, billing issues temporarily rise to 50%. The model has not changed, but one part of the environment has: the prior probability of each class.

Artificial Intelligence Machine Learning

Artificial Intelligence 04 Sep 2026 10 min read

Detect Distribution Shift Before Model Quality Fails

A model can pass offline evaluation and still become less useful after deployment. The model may not have changed at all. Instead, the data reaching it may have changed. A fraud classifier trained on last year’s transactions may encounter a new payment pattern. A support-ticket model may see terminology introduced by a new product. An image model deployed to different hardware may receive images with different lighting or compression. These are forms of distribution shift: the statistical conditions seen in production differ from those represented by the data used to develop or evaluate the model.

Artificial Intelligence Machine Learning