00:00 AM

Pitt research team part of $10 million NSF grant to develop Visual Cortex on Silicon

PITTSBURGH  (September 17, 2013) … Developing a computer that can see the world like the human brain's complex visual cortex has been a long-sought challenge. Thanks to a $10 million grant from the National Science Foundation's (NSF) Directorate for Computer and Information Science and Engineering (CISE), that challenge may be a future reality. 

A University of Pittsburgh research team is part of a seven-university group led by Principle Investigator  Vijaykrishnan Narayanan, PhD , Professor of Computer Science and Engineering at The Pennsylvania State University, that will receive an NSF CISE Expeditions in Computing awards, the largest single investment in computer science research that NSF makes.

The Pitt team, which will be funded by $500,000 of the total grant, is led by  Steven P. Levitan, PhD , the John A. Jurenko Professor of Computer Engineering at the Swanson School of Engineering; and  Donald M. Chiarulli, PhD , Professor of Computer Engineering and Computer Science in Pitt's Department of Computer Science. Student researchers are undergraduates Soyo Awosika-Olumo, Natalie Janosek and Andrew Seel; and graduate students John Carpenter and Yan Fang. Other collaborating institutions include University of Southern California, Stanford University, York College of Pennsylvania, University of California-San Diego, University of California-Los Angeles, and Massachusetts Institute of Technology.

According to the NSF's announcement, this project envisions a holistic design of a machine vision system that will approach or exceed the capabilities and efficiencies of human vision, enabling computers to not only record images, but also to understand visual content, at up to a thousand times the efficiency of current technologies.

While several machine vision systems today can each successfully perform one or a few human tasks - such as detecting human faces in point-and-shoot cameras - they are still limited in their ability to perform a wide range of visual tasks, to operate in complex, cluttered environments, and to provide reasoning for their decisions. In contrast, the visual cortex in mammals excels in a broad variety of goal-oriented cognitive tasks, and is at least three orders of magnitude more energy efficient than customized state-of-the-art machine vision systems.

In particular, this Expedition aims to understand the fundamental mechanisms used in the visual cortex, with the hope of enabling the design of new vision algorithms and hardware fabrics that can improve power, speed, flexibility, and recognition accuracies relative to existing machine vision systems.

Because of the complexity of the human visual cortex, which processes image data faster than a computer, the teams are applying their expertise across multiple systems. Pitt's Levitan and Chiarulli are investigating the resonance of coupled electro-magnetic oscillators to perform image processing in silicon. "The underlying principle is not new - in fact, in 1665, Christiaan Huygens discovered that two pendulum clocks mounted to the same wall would synchronize their swings, no matter how they were started," Dr. Levitan explains. "This self-synchronization property of coupled oscillators can be used as a measure of similarity in large scale pattern matching problems."

Electro-magnetic spin torque oscillators are an emerging nano-technology that is fabricated as a sandwich of two magnetic films separated by a non-magnetic layer. When an electric current passes through the sandwich the electrons cause the magnetic field to rotate like a spinning top, which creates an oscillating voltage. The oscillators can be coupled so they resonate when they are at the same frequency. Levitan and Chiarulli are exploiting this locking phenomenon to match patterns in visual images. "Imagine a vast sea of organ pipes, each tuned to one frequency. If you then play a note in the air, the pipe tuned closest to that note will resonate. If we identify the pipe, we know which note it was; that is the basic idea."

Dr. Chiarulli adds, "Traditional Boolean logic - the 1s and 0s or true/false values that comprise computer systems - is not capable of the nuanced pattern matching and image analysis that the human brain does many times per second using less than ten watts of power. This work is focused on new computational models that can work more like a human brain model of computation in solving the problem of visual perception."

Smart machine vision systems that understand and interact with their environments could have a profound impact on society, including aids for visually impaired persons, driver assistance capabilities for reducing automotive accidents, and augmented reality systems for enhanced shopping, travel, and safety.

"Advances in the computer and information sciences drive progress in all areas of science, engineering and education, which positively impacts the U.S. economy, furthers national priorities and bolsters our overall quality of life," said NSF Acting Director Cora Marrett. "America's future depends on strong and sustained U.S. government support in this area. NSF is proud to fund this next round of Expeditions awards, and in supporting fundamental research, to continue its tradition of enabling the nation to maintain its competitive advantage in information technology."

The first Expeditions awards were made in 2008. As of today, 16 awards have been made through this program, addressing subjects ranging from foundational research in computing hardware, software and verification to research in sustainable energy, health information technology, robotics, mobile computing, and Big Data.

"The Expeditions in Computing program catalyzes large-scale, far-reaching and potentially transformative research motivated by deep scientific questions," said Farnam Jahanian, assistant director for CISE. "These two new awards aim to apply our understanding of natural, biological capabilities to the development of revolutionary new computing and information technologies with tremendous potential for societal benefit."

"Without exception, the Expeditions in Computing awards fund bold, ambitious and exciting research," said Mitra Basu, program director for the Expeditions program. "We are now seeing how they advance the field and lead to exciting results in a variety of applications. We're confident that these two projects have the same potential for pushing the frontiers of computing."


The NSF news release may be found here.

Contact: Paul Kovach