ShopSphere AICV tracks how a single platform algorithm update rippled into a sharp rise in cancellations — and the TabTransformer model built to catch it early.
AICV compares the historical churn baseline against churn observed after the ShopSphere algorithm update, isolating the size of the shock.
The post-update churn rate increased substantially compared with the historical baseline, highlighting the business impact associated with the platform algorithm change.
Longitudinal synthetic customer data created to model ShopSphere customer behavior before and after the algorithm update.
A custom TabTransformer-style neural network was trained to classify customer churn using behavioral, engagement, visibility, pricing and satisfaction features. Metrics below are from a cost-based decision threshold, not the default 0.50 — see why underneath.
The decision threshold is chosen to maximize expected retained value, not accuracy — since missing a churner costs an assumed 25× more than an unnecessary retention outreach, the optimal policy flags a much wider net of customers. That trades accuracy for recall on purpose. At a precision of 12.9%, roughly 7 out of every 8 flagged customers are false alarms — a real deployment would need to weigh that against how many customers a retention team can actually reach.
Logistic Regression and Histogram Gradient Boosting, trained on identical data splits and evaluated with the same cost-based threshold logic.
| Model | Accuracy | Precision | Recall | F1 | ROC-AUC | Threshold |
|---|---|---|---|---|---|---|
| Logistic Regression | 50.9% | 14.0% | 81.8% | 23.9% | 73.5% | 0.35 |
| Gradient Boosting | 14.5% | 9.9% | 98.8% | 17.9% | 68.2% | 0.02 |
| TabTransformer (deployed) | 44.6% | 12.9% | 84.7% | 22.4% | 73.3% | 0.17 |
Logistic Regression matches or slightly exceeds the TabTransformer on precision, F1, and ROC-AUC. On a dataset this size (1,000 customers), that's expected — the added architectural complexity isn't earning its keep here. The TabTransformer was retained for this project because it was the assigned architecture, not because the comparison favors it. Gradient Boosting's threshold collapsed to near-zero, flagging almost every customer — a sign its probabilities aren't well-calibrated out of the box, not a genuinely useful policy.
Loading the trained model weights (~1 MB) for real in-browser inference…
Every field above feeds the actual trained TabTransformer weights via ONNX Runtime, running in your browser. The 16 numerical features not shown here are held at their training-set average for each prediction.
The model supports retention prioritization by balancing the cost of unnecessary interventions against the potential value lost when a churner is missed.
Customers with elevated predicted churn probability can be prioritized for retention interventions. The 0.17 decision threshold maximizes expected net value under the ₹200-vs-₹5,000 cost assumption — but as shown above, at this precision (12.9% on the real held-out test set) most flagged customers will be false alarms. That's a deliberate tradeoff for this cost ratio, not a flaw to hide, and any real deployment should weigh it against retention-team capacity.
Synthetic customers, scored by the real model. Flag status updates live as you move the threshold above — click any column to re-sort.
| Customer | Tenure (mo) | Visibility | Satisfaction | Segment | Churn risk | Status |
|---|
This project uses a synthetic dataset created for academic demonstration. The observed relationship between the algorithm update and churn should therefore not be interpreted as causal evidence from a real commercial platform.
Model performance should be evaluated further on real-world customer data before deployment in an operational retention system.