Question 1
A financial services firm has deployed a credit default prediction model into production using Watson Machine Learning. The model was trained on data from the past five years. After six months in production, the model's performance, monitored via Watson OpenScale, shows a significant drop in accuracy and a drift in the distribution of key features like 'debt-to-income ratio' and 'number of open credit lines'. The MLOps team needs to devise a strategy to address this issue.
What is the most appropriate first step to diagnose and mitigate this problem?
Answer and explanation
Correct answer: B
The problem described is a classic case of concept drift, where the statistical properties of the target variable change over time. Simply retraining the model without understanding the cause is inefficient and may not solve the underlying problem. The best first step is to use the monitoring tools (Watson OpenScale) to diagnose the issue, identify the specific features causing the drift, and collaborate with business experts to understand the 'why' behind the change (e.g., new economic policies, a shift in consumer behavior). This informed approach leads to a more robust and lasting solution, such as feature re-engineering or adopting an adaptive learning strategy.