Pitt Innovation Challenge awards $10K to Bioengineering's George Stetten for FingerSight / PalmSight wearable technology

PITTSBURGH (November 5, 2017) - FingerSight is a project that was originally developed in 1999 by George Stetten, professor of bioengineering at the University of Pittsburgh, to help blind and visually impaired individuals better perceive their surroundings. 

Dr. Stetten explains, “My father went blind, and I always had an interest in developing technologies to help those with this disability.” 

Dr. Stetten and Roberta Klatzky, professor of psychology at Carnegie Mellon University, patented the concept of the device, with a camera mounted on the fingertip and a vibrating mechanism on the same finger. This allows the user to quickly scan the environment while computer vision algorithms to find objects identified by verbal command.  Detection of the object is communicated through the vibrator. 

The original version of the device had a laser that scanned the edges of objects. This project was featured on the cover of IEEE Haptics Symposium in 2006. Dr. Stetten’s group later improved upon that model by researching how the camera can sense the angles of the object edges. This work was published in the Journal of Translational Engineering in Health and Medicine in 2014.  

This device uses computer vision, a powerful and quickly evolving discipline in which information about the environment is received by video camera and interpreted by computer.

Some practical applications of the device include reading street signs, locating exits, finding particular people, or avoiding street hazards.

The device also lets the user control objects on a screen or intelligent targets through gestures. 

Dr. Stetten’s newest project is PalmSight, which is a device that helps the user reach for an object and grasp it. It uses stereo cameras mounted to the user’s palm to determine depth and an array of vibrators to guide hand motion. The user can ask for a specific object (e.g. a coffee cup), and the device will help guide the user’s hand to that object.

Dr. Stetten will use the PInCh funds to further develop this technology towards commercialization.