Data Scientist Capstone: From Model to Decision

Programme overview

The capstone of the Data Scientist track. Kasuwa, an online shop, loses about ₦40m a year to pay-on-delivery orders that fail at the door, and its head of operations wants "a model that stops bad orders". You'll frame it as a decision (whom to phone before dispatch), audit 67,000 orders for leaks, build customer history as it was known at checkout, and validate models over time, testing only on orders the trial didn't touch. Then you'll check calibration, use a randomised trial of confirmation calls to measure what a call actually prevents at each level of risk, and set the threshold where calls pay for themselves. Finally, you'll check the model city by city (it badly misses Kaduna, a market it has never seen), decide whether it should use location at all, write the model card, and plan monitoring that would catch the problems a single drift number hides. Every number comes from running the code.

Curriculum

1. 01The Problem and the Plan25 min

   • The problem and the planMeet Kasuwa, an online shop losing money on pay-on-delivery orders that fail at the door. Turn "build a model to stop bad orders" into a decision, a target, a prediction moment and a metric in naira.
2. 02Audit the Data25 min

   • Audit the dataSort every column into known at checkout or known later, find the leaks before they find you, and explore what's linked to failed deliveries.
3. 03Point-in-Time Features30 min

   • Point-in-time featuresBuild the customer's history as it was at checkout, including only failures that had already happened. Prove the difference against the tempting shortcut and the leaky lifetime totals.
4. 04Validate over Time30 min

   • Validate over timeSplit by time, not at random, so the model is tested the way it will be used. Compare a simple rule, logistic regression and gradient boosting, and test on orders the trial didn't touch.
5. 05Calibration and Capacity25 min

   • Calibration and capacityCheck that a predicted 30% risk really means about 30%, see how many failures the riskiest orders contain, and find out why ranking alone can't tell you whom to call.
6. 06The Trial and Who to Call30 min

   • The trial and who to callUse the randomised April trial to measure what a confirmation call achieves, find out how the effect grows with risk, and turn the model into a calling policy with a threshold set in naira.
7. 07Fairness and Explanation30 min

   • Fairness and explanationCheck how the model performs and who it selects in each city, find out what drives its predictions, test what happens without the city feature, and write the model card.
8. 08Monitoring and the Final Presentation30 min

   • Monitoring and the final presentationPlan how to tell when the model stops working: calibration by month, score drift and new markets. Then present the project to Kasuwa's leadership as a decision, with its value, its limits and its safeguards.

Assessment and certificate

You are assessed with a final assessment (pass mark 60%) and a final project. Your certificate is awarded when you meet those requirements. Paying for the programme does not award it on its own.