Concerns That AI-Driven Incident Response Automation May Hinder Engineer Proficiency

However, he also expressed significant concerns. He pointed out that as opportunities to handle routine incidents decrease, engineers may lose the chance to develop intuition regarding system behavior and failure patterns.

This issue mirrors the "ironies of automation" described in a 1983 paper, where increased automation leads to operators losing training opportunities for routine tasks, while their responsibility for handling abnormal situations becomes heavier. When complex and unknown events occur that AI cannot resolve, there is a possibility that engineer response times will be delayed.

He predicted that while the use of AI will shorten the Mean Time to Recovery (MTTR) for many incidents, the resolution time for complex problems may increase significantly due to a decline in investigation capabilities.

To address this challenge, Rootly, the company he belongs to, has introduced simulation training utilizing LLMs (Large Language Models). Efforts are underway to maintain and improve skills that are difficult for AI to replace—such as decision-making and coordination under incomplete information—by having engineers perform simulated fault responses.