Statistical Model Lifecycle Management

Approach

The solution was developed based on the concepts of Statistical Quality Control esp. Western Electric rules. These are decision rules for detecting “out-of-control” or non-random conditions using the principle of process control charts. Distributions of the observations relative to the control chart indicate whether the process in question should be investigated for anomalies.

  1. Any single data point falls outside the 3σ limit from the centerline (i.e., any point that falls outside Zone A, beyond either the upper or lower control limit)
  2. Two out of three consecutive points fall beyond the 2σ limit (in zone A or beyond), on the same side of the centerline
  3. Four out of five consecutive points fall beyond the 1σ limit (in zone B or beyond), on the same side of the centerline
  4. Eight consecutive points fall on the same side of the centerline (in zone C or beyond)

Business Case

A large beverage company wanted to forecast industry-level demand for a specific product segment in multiple sales geographies. Affine evaluated multiple analytical techniques and identified a champion model based on accuracy, robustness, and scalability. Since the final model was supposed to be owned by client internal teams, Affine enabled assessing the lifecycle stage of a model through an automated process. A visualization tool was developed which included an alert system to help users proactively identify any red flags. A detailed escalation mechanism was outlined to address any queries or red flags related to model performance or accuracies.

  1. Two out of three consecutive points fall beyond the 2σ limit (in zone A or beyond), on the same side of the centerline
  2. Four out of five consecutive points fall beyond the 1σ limit (in zone B or beyond), on the same side of the centerline
  3. Eight consecutive points fall on the same side of the centerline (in zone C or beyond)

Key Impact and Takeaways

  1. Quantify and develop benchmarks for error limits.
  2. A continuous monitoring system to check if predictive model accuracies are within the desired limit.
  3. Prevent undesirable escalations thus rationalizing operational costs.
  4. Enabled through a visualization platform. Hence does not require strong analytical
    expertise.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Affine

Affine

104 Followers

Affine is a provider of analytics solutions, working with global organizations solving their strategic and day to day business problems https://affine.ai