Expert Viewpoint: The Intersection Between VR, Imaging and Machine Learning and What’s Next for the Emerging Technologies

June 6, 2018

Ramesh Raskar is Director of the MIT Media Lab’s Camera Culture research group. He previously worked as a Senior Research Scientist at Mitsubishi Electric Research Laboratories and is currently teaching a course titled, “Advances in Imaging: VR-AR, Machine Learning, and Self-Driving Cars”. He recently shared his views on the implications of the rapidly evolving technologies of imaging, virtual reality, and machine learning.

R&D Magazine: How well does the average non-technical person understand imaging technology?

Raskar: This is a challenge. Imaging is a hot 21st Century topic, along with machine learning and virtual reality. It’s somewhat accessible to anyone who uses a smartphone or puts on a virtual reality headset. We’re basically talking about cameras—advanced, highly-developed cameras and sensing devices. One important use for them is in self-driving cars, another hot topic. Engineers and laypeople alike ought to be interested in exploring their present and future practical applications.

R&D Magazine: In your estimate, which industries should be interested in imaging technology?

Raskar: Venture capitalists, business start-ups that are tech-centric, retail. Also, people from research laboratories such as Lincoln Labs, National Labs, and the government’s official labs. Members of all those groups need to stay abreast of new developments.

R&D Magazine: How is imaging technology combined with machine learning to develop and program the self-driving car? Where are we currently at with that advancement?

Raskar: It’s a matter of getting a computer, with the help of the imaging camera, to recognize objects ahead. Is that a pedestrian or not? Is that a cat or a dog in front of you? Is that a stop sign or not? That’s easy enough for humans to do, but very difficult for computers.  However, the problem is being solved through machine learning—teaching the computer to recognize those things. MIT researchers developed a system that can produce images of objects shrouded by fog so thick that human vision can’t penetrate it. It can also gauge the objects’ distance. That’s a huge transformation.

R&D Magazine: What other transformative breakthroughs to you foresee in these industries?

Raskar: Imaging technology is well down the road to revolutionizing life and commerce. Most visible would be the entertainment and tourism industries where just putting on a headset or other high-end imaging device puts you—virtually—in a given place. Other potential applications are numerous and multiplying.

More deeply technical, but also very promising is the process known as compressed sensing, in which MIT scientists have also played a pivotal role. Researchers at Rice University built a camera that could produce 2D images using only a single light sensor rather than the millions of light sensors found in a commodity camera.

However, the process was inefficient. That “single-pixel camera” needed thousands of exposures to produce a reasonably clear image. So researchers from the MIT Media Lab arrived at a new technique 50 times as efficient. In the case of the single-pixel camera, it could get the number of exposures down from thousands to dozens. This is remarkable. And, again, it is known as compressed sensing.

R&D Magazine: What are the major trending imaging tech areas?

Raskar: I would say self-driving cars, machine learning, and advances in camera photography. Cell phone companies often advertise their device’s ability to take pictures more than its capacity for answering calls. Then you have virtual reality’s link with automation. This has practical implications for the economy, including manufacturing where imaging technology can be used, to among other purposes, operate things like conveyor belts.

R&D Magazine: Is there a lot of investment in this area?  

Raskar: Google, to name just one tech giant, has bought three or four virtual reality companies for about a billion each. There are tons of jobs being generated in this area. This is why CEOs should be eager to learn all they can about it. Imaging cameras are everywhere. I would say to those CEOs, what is your imaging strategy for the future that will create a competitive advantage for you?

R&D Magazine: Do you see any problems as this technology expands?

Raskar: Well, we need to democratize it. In the tech world, there is a saying, “Capture, analyze, experience.”  This also goes for the general public. Too many lack proficiency with these important tools that have the capacity to improve their lives, which is where education comes in. About fifty people come to the MIT course I teach. I’d like to double that to help people keep up with a fast-moving world.