By Carolyn Mathas, Contributor
Deep neural networks, although sophisticated, can be fooled. When this happens the results can be deadly if the network is guiding an autonomous vehicle. But what is the source of that failure?
A recent paper published in Nature Machine Intelligence asserts that vulnerabilities are due to susceptibility to adversarial examples. The author suggests that the networks may be detecting features that are “predictively useful, yet inscrutable to humans.” Cameron Buckner, associate professor of philosophy at UH, says that it is critical to understand the source of failures caused by what researchers call “adversarial examples,” when a deep neural network system misjudges images or data outside the training inputs that were used to build the network.
The misfire could be caused by the interaction between what the network is asked to process and the actual patterns involved. Buckner proposes that some of these patterns are artifacts.
The highest risk in adversarial events that cause the system to make errors are caused by intentional malfeasance. For example, a security system based upon facial recognition technology can be hacked to allow a breach or decals could be placed on traffic signs causing self-driving cars to misinterpret the sign.
There are also naturally occurring adversarial examples, when a machine learning system misinterprets data through an unanticipated interaction, suggesting the need to rethink how researchers approach anomalies or artifacts. For example, a lens flare in a photograph isn’t caused by a defect in the camera lens but is instead produced by the interaction of light with the camera. The information the flare provides, such as the location of the sun, could be useful. Buckner maintains that this new way of thinking about how artifacts can affect deep neural networks suggests a misreading by the network and is not evidence that deep learning isn’t valid.
