Small stickers on the ground trick Tesla autopilot into opposing traffic lane.

Tesla Cockpit (Source Tencent Report)

05/05/2019

Do we understand AI and do we trust AI? These were the two basic questions raised by Katherine Jarmul during her presentation at the Being Human with Algorithms symposium in Heidelberg last year. Her answer was we should be extremely careful, because we are still in an early stage before machine learning systems may reliably cope with the real world. There are patterns which could be easily recognized by humans but still pose a problem to AI systems. The literature is full with adversarial learning examples where turtles were recognized as guns, and little patches on stop signs where interpreted as go-faster signs. The latest example is a recent attack to the auto-steering system of a Tesla Model S 75.

Researchers from Tencent Keen Security Lab have published a report detailing their successful attacks on Tesla firmware, including remote control over the steering, and an adversarial example attack on the autopilot that confuses the car into driving into the oncoming traffic lane.

The researchers used an attack chain that they disclosed to Tesla, and which Tesla now claims has been eliminated with recent patches.

To effect the remote steering attack, the researchers had to bypass several redundant layers of protection, but having done this, they were able to write an app that would let them connect a video-game controller to a mobile device and then steer a target vehicle, overriding the actual steering wheel in the car as well as the autopilot systems. This attack has some limitations: while a car in Park or traveling at high speed on Cruise Control can be taken over completely, a car that has recently shifted from R to D can only be remote controlled at speeds up to 8km/h.

Tesla vehicles use a variety of neural networks for autopilot and other functions (such as detecting rain on the windscreen and switching on the wipers); the researchers were able to use adversarial examples (small, mostly human-imperceptible changes that cause machine learning systems to make gross, out-of-proportion errors) to attack these.

Most dramatically, the researchers attacked the autopilot’s lane-detection systems. By adding noise to lane-markings, they were able to fool the autopilot into losing the lanes altogether, however, the patches they had to apply to the lane-markings would not be hard for humans to spot.

Much more seriously, they were able to use “small stickers” on the ground to effect a “fake lane attack” that fooled the autopilot into steering into the opposite lanes where oncoming traffic would be moving. This worked even when the targeted vehicle was operating in daylight without snow, dust or other interference.

[Total: 0    Average: 0/5]

Avatar

Gerhard Schimpf, the recipient of the ACM Presidential Award 2016, has a degree in Physics from the University of Karlsruhe. As a former IBM development manager and self-employed consultant for international companies, he has been active in ACM for over four decades. He was a leading supporter of ACM Europe, serving on the first ACM Europe Council in 2009. He was also instrumental in coordinating ACM’s spot as one of the founding organizations of the Heidelberg Laureates Forum. Gerhard Schimpf is a member of the German Chapter of the ACM (Chair 2008 – 2011) and a member of the Gesellschaft für Informatik. --oo-- Gerhard Schimpf, der 2016 mit dem ACM Presidential Award geehrt wurde, hat an der TH Karlsruhe Physik studiert. Als ehemaliger Manager bei IBM im Bereich Entwicklung und Forschung und als freiberuflicher Berater international tätiger Unternehmen ist er seit 40 Jahren in der ACM aktiv. Er war Gründungsmitglied des ACM Europe Councils und gehört zum Founders Club für das Heidelberg Laureate Forum, einem jährlichen Treffen von Preisträgern der Informatik und Mathematik mit Studenten. Gerhard Schimpf ist Mitglied des German Chapter of the ACM (Chairperson 2008 – 2011) und der Gesellschaft für Informatik.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

WP2Social Auto Publish Powered By : XYZScripts.com