美国安全与新兴技术中心2024人工智能安全与自动化偏见人机交互的风险研究报告英文版34页

Issue BriefNovember 2024AI Safety and Automation BiasThe Downside of Human-in-the-LoopAuthors Lauren Kahn Emelia S. Probasco Ronnie KinoshitaCenter for Security and Emerging Technology | 1 Executive Summary Automation bias is the tendency for an individual to over-rely on an automated system. It can lead to increased risk of accidents, errors, and other adverse outcomes when individuals and organizations favor the output or suggestion of the system, even in the face of contradictory information. Automation bias can endanger the successful use of artificial intelligence by eroding the user’s ability to meaningfully control an AI system. As AI systems have proliferated, so too have incidents where these systems have failed or erred in various ways, and human users have failed to correct or recognize these behaviors. This study provides a three-tiered framework to understand automation bias by examining the role of users, technical design, and organizations in influencing automation bias