Eyes on the road. When talking about driver safety, that’s a statement people have heard for years.
Electrical and Computer Engineering Professor Yi Lu Murphy is working on a project that takes those words of the past and applies it to artificial intelligence of the future.
In her Intelligent Systems Lab, Murphey is working to perfect a car with 360-degree vision.
“We want the car to continuously see; to have a view of the road at all times. Through our research, we have a computer vision system that can detect the traffic signs, like stop signs and yield signs. Now we are moving toward accurately detecting pedestrians,” said Murphey, who is the College of Engineering and Computer Science (CECS) associate dean for Graduate Education and Research.
The project is a collaboration with ZF, formerly known at TRW Automotive, one of the world’s largest automotive suppliers.
Murphey and the students in her lab use data gathered from a technologically advanced vehicle, which has four outwardly facing cameras and sensors—in the front, rear and on the left and right door mirrors.
The goal is a 360-degree view around the car that allows the team to better assess their detection accuracy research through real-world data gathering. The vehicle is driven on campus, around Detroit and through downtown Ann Arbor.
“Right now, depending on the light, color shading and movement, pedestrian detection is a challenge for vehicle sensors. But our research wants to change that and make the vehicle’s vision just as good—if not better—than a human’s,” said Murphey, who has worked with artificial intelligence systems for more than 30 years, with half of that time specifically researching vehicle safety systems. “Once that vision is developed, it can be implemented into safety features.”
CECS student Chengqi Bian, a graduate research assistant, and Zihao Zhao, an undergraduate research assistant, recently took a ride around campus in the ZF’s black Nissan Rogue, recording the campus life around them with the 360-degree computer vision system.
Zhao, in the passenger seat, watched four-camera footage on his laptop, remarking that every 10 minutes the recordings are saved, so that the files don’t become too large to work with. After the data is collected, the students will return to the lab and review and tag what’s recorded—signs, people, cars, etc.—to go into a large database of tagged images with the goal of teaching, through an algorithm, the computer system to accurately identify objects.
Both students said they knew of Murphey’s research before coming to campus—Zhao is interested in data science research and Bian is interested in computer vision systems—and are pleased that they were given the opportunity to work in Murphey’s lab.
“Dr. Murphey is a leading researcher who is very well respected around the world. I was familiar with her research and wanted to work in her lab,” said Bian, who earned his undergraduate degree in electrical engineering from Xi’an Jiaotong University in China. “I’ve worked in the lab for almost two years and we are seeing great improvement with the deep learning computer vision system model.”
In addition to the 360-degree computer vision project, Murphey has looked at a multitude of ways to improve advanced driver vehicle assistance systems with ZF in Northville, Mich., over the past decade.
Through the partnership, the Intelligent Systems Lab has received annual research education grants and has built, tested and evaluated several systems for a variety of projects.
For example, they created a computer vision system for airbags to detect who is sitting in the passenger seat—child or adult—and, depending on the individual type and crash characteristics tailor the airbag deployment to the current situation in order to protect the passenger. They also have researched driver fatigue detection and found that evaluating scenarios such as lane changes in high-quality real-time video can help accurately detect drowsiness.
For ZF, investing in university projects can help guide the industry over the paradigm shift—conventional vehicles to autonomous—happening in the auto industry, said Franz Kleiner, member of the Board of Management of ZF Friedrichshafen AG.
“The challenges of the next stages of mobility are both exciting and at times daunting, and we need brilliant young minds to bring fresh perspectives,” he said. “Partnering with leading Michigan universities, like the University of Michigan-Dearborn, allows us to explore new ideas at a faster pace, and it can give students opportunities to contribute to the future of transportation.”
Murphey said the future is autonomous vehicles, but—while we wait for those to become available—the 360-degree computer vision system has present-day application, as well.
She said even good drivers of conventional vehicles get too busy and make mistakes. And an intelligent vision system like this may help people stay safely on the road longer.
“That is where the camera and sensors come in. If they can detect something or someone on the road—when you don’t—the system can tell the vehicle to stop. There is obvious use for this in autonomous vehicles, but it also has an important use in today’s conventional vehicles,” she said. “We want a system that complements the driver; one that sees what the driver may not so that everyone on the road, even the cats and squirrels, stay safe.”
Safety is the top priority behind Murphey’s work, but seeing her students—like Bian and Zhao—gain opportunities through research and industry partnerships is rewarding too.
“This is a way for our students to understand the complexity of engineering problems and help them learn how modern technologies can solve a problem that has societal impact,” she said. “Making advanced safety features for vehicles is important work; it saves lives.”
