Driving certificate

Making Self-Driving Cars Safer Through More Accurate Robot Perception | MIT News

Aviation became a reality at the start of the 20th century, but it took 20 years before proper safety measures allowed widespread adoption of air travel. Today, the future of fully autonomous vehicles is equally murky, largely due to safety concerns.

To speed up this timeline, graduate student Heng “Hank” Yang and his collaborators have developed the first set of “certifiable perception” algorithms, which could help protect the next generation of autonomous vehicles – and the vehicles they share with. the road.

Although Yang is now a rising star in his field, it took many years before he decided to research robotics and autonomous systems. Raised in China’s Jiangsu Province, he earned his undergraduate degree with highest honors from Tsinghua University. His time at university was spent studying everything from bees to cell mechanics. “My curiosity pushed me to study a lot of things. Over time, I started to drift more into mechanical engineering because it cuts across so many other fields,” Yang explains.

Yang then pursued a master’s degree in mechanical engineering at MIT, where he worked on improving an ultrasound imaging system to track liver fibrosis. To achieve his engineering goal, Yang decided to take a course on designing algorithms to control robots.

“The course also covered mathematical optimization, which is adapting abstract formulas to model almost anything in the world,” says Yang. “I learned an interesting solution to iron out the final details of my thesis. I was amazed at how powerful computation can be in optimizing the design. From there, I knew it was the right area to explore next.

Algorithms for certified accuracy

Yang is now a graduate student at the Laboratory for Information and Decision Systems (LIDS), where he is working with Luca Carlone, Leonardo Career Development Associate Professor of Engineering, on the challenge of certifiable perception. When robots sense their environment, they must use algorithms to make guesses about the environment and their location. “But these perception algorithms are designed to be fast, with little guarantee that the robot has succeeded in gaining a correct understanding of its environment,” says Yang. “It’s one of the biggest problems out there. Our lab is working on designing “certified” algorithms that can tell you if these estimates are correct. »

For example, robot perception begins with the robot capturing an image, like a self-driving car taking a snapshot of an approaching car. The image goes through a machine learning system called a neural network, which generates key points in the image regarding mirrors, wheels, doors, etc. of the approaching car. From there, lines are drawn to search for keypoints detected on the 2D car image to 3D keypoints labeled in a 3D car model. “We then need to solve an optimization problem to rotate and translate the 3D model to align it with key points in the image,” Yang explains. “This 3D model will help the robot understand the real environment.”

Each line drawn should be analyzed to see if it created a correct match. Since there are many key points that could be mismatched (e.g. the neural network could mistakenly recognize a mirror as a doorknob), this problem is “non-convex” and difficult to solve. Yang says his team’s algorithm, which won the best paper on robotic vision at the International Conference on Robotics and Automation (ICRA), smoothes the nonconvex problem to become convex and finds successful matches. “If the match is not correct, our algorithm will know to keep trying until it finds the best solution, known as the global minimum. A certificate is issued when there are no better solutions,” he explains.

“These certifiable algorithms have huge potential impact, as tools like self-driving cars need to be robust and reliable. Our goal is to ensure that a driver receives an alert to take the wheel in the event of a fault in the perception system.

Adapt their model to different cars

When matching the 2D image to the 3D model, one assumption is that the 3D model will align with the identified car type. But what if the pictured car has a shape that the robot has never seen in its library? “We now need to both estimate the position of the car and reconstruct the shape of the model,” says Yang.

The team found a way to navigate around this challenge. The 3D model is transformed to match the 2D image by undergoing a linear combination of previously identified vehicles. For example, the model could change from an Audi to a Hyundai because it saves the correct version of the real car. Identifying the dimensions of the approaching car is key to preventing collisions. This work earned Yang and his team finalists for the Best Paper Award at the Robotics: Science and Systems (RSS) conference, where Yang was also named an RSS Pioneer.

In addition to presenting at international conferences, Yang enjoys discussing and sharing his research with the general public. He recently shared his work on certifiable perception at MIT’s first public SLAM research showcase. He also co-hosted the first virtual LIDS student conference alongside industry leaders. His favorite lectures have focused on ways to combine theory and practice, such as Kimon Drakopoulos’ use of AI algorithms to guide how to allocate Greece’s Covid-19 testing resources. “What stood out to me was how he really focused on what these rigorous analytical tools can do to benefit social good,” Yang says.

Yang plans to continue researching difficult issues that deal with safe and trustworthy autonomy by pursuing a career in academia. His dream of becoming a professor is also fueled by his love of mentoring, something he enjoys doing in Carlone’s lab. He hopes his future work will lead to further discoveries that will help protect people’s lives. “I think many realize that the set of solutions we have to promote human security is not enough,” Yang says. “In order to achieve trustworthy autonomy, it’s time for us to embrace a diverse set of tools to design the next generation of secure perception algorithms.”

“There must always be built-in security, because none of our man-made systems can be perfect. I believe it will take both the power of rigorous theory and calculation to revolutionize what we can successfully reveal to the public.