Identifying Perturbed Roadway Signs: Perception of AI Capabilities
Event Type
TimeTuesday, October 11th4:42pm - 5:00pm EDT
DescriptionArtificial Intelligence (AI) is crucial to numerous functions required for driving automation systems, including the computer vision techniques used to detect the roadway environment to make real-time decisions. However, these images may be maliciously perturbed, or manipulated, causing the AI to make an incorrect classification. We examined how humans perceive AI’s capability of identifying various road-sign images, including the original images, images with two types of malicious attacks, and scrambled images. Our results showed that participants rated themselves to be more capable than the AI of identifying the road signs except when the images were randomly scrambled. However, they overestimated the AI’s capability for correctly identifying images with malicious attacks that should cause the AI to misclassify the image. These findings suggest that people lack an accurate understanding of the vulnerabilities of AI computer vision techniques and tend to overtrust the AI in driving automation systems.