Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:


Dartmouth team creates first hidden, real-time, screen-camera communication


Opening the way for new applications of smart devices, Dartmouth researchers have created the first form of real-time communication that allows screens and cameras to talk to each other without the user knowing it.

Using off-the-shelf smart devices, the new system supports an unobtrusive, flexible and lightweight communication channel between screens (of TVs, laptops, tablets, smartphones and other electronic devices) and cameras. The system, called HiLight, will enable new context-aware applications for smart devices.

Dartmouth researchers have created the first form of real-time communication that allows screens, displaying images such as this landscape, and cameras to talk to each other without the user knowing it.

Credit: Dartmouth College

Such applications include smart glasses communicating with screens to realize augmented reality or acquire personalized information without affecting the content that users are currently viewing. The system also provides far-reaching implications for new security and graphics applications.

The findings will be presented May 20 at the ACM MobiSys'15, a top conference in mobile systems, applications and services. A PDF of the study, further information and demonstration videos are available at the HiLight project website.

In a world of ever-increasing smart devices, enabling screens and cameras to communicate has been attracting growing interest. The idea is simple: information is encoded into a visual frame shown on a screen, and any camera-equipped device can turn to the screen and immediately fetch the information.

Operating on the visible light spectrum band, screen-camera communication is free of electromagnetic interference, offering a promising alternative for acquiring short-range information. But these efforts commonly require displaying visible coded images, which interfere with the content the screen is playing and create unpleasant viewing experiences.

The Dartmouth team studied how to enable screens and cameras to communicate without the need to show any coded images like QR code, a mobile phone readable barcode. In the HiLight system, screens display content as they normally do and the content can change as users interact with the screens. At the same time, screens transmit dynamic data instantaneously to any devices equipped with cameras behind the scene, unobtrusively, in real time.

HiLight supports communication atop any screen content, such as an image, movie, video clip, game, web page or any other application window, so that camera-equipped devices can fetch the data by turning their cameras to the screen. HiLight leverages the alpha channel, a well-known concept in computer graphics, to encode bits into the pixel translucency change. HiLight overcomes the key bottleneck of existing designs by removing the need to directly modify pixel color values. It decouples communication and screen content image layers.

"Our work provides an additional way for devices to communicate with one another without sacrificing their original functionality," says senior author Xia Zhou, an assistant professor of computer science and co-director of the DartNets (Dartmouth Networking and Ubiquitous Systems) Lab. "It works on off-the-shelf smart devices. Existing screen-camera work either requires showing coded images obtrusively or cannot support arbitrary screen content that can be generated on the fly. Our work advances the state-of-the-art by pushing screen-camera communication to the maximal flexibility."


Assistant Professor Xia Zhou is available to comment at

Broadcast studios: Dartmouth has TV and radio studios available for interviews. For more information, visit:

Media Contact

John Cramer


John Cramer | EurekAlert!

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: New 3-D wiring technique brings scalable quantum computers closer to reality

Researchers from the Institute for Quantum Computing (IQC) at the University of Waterloo led the development of a new extensible wiring technique capable of controlling superconducting quantum bits, representing a significant step towards to the realization of a scalable quantum computer.

"The quantum socket is a wiring method that uses three-dimensional wires based on spring-loaded pins to address individual qubits," said Jeremy Béjanin, a PhD...

Im Focus: Scientists develop a semiconductor nanocomposite material that moves in response to light

In a paper in Scientific Reports, a research team at Worcester Polytechnic Institute describes a novel light-activated phenomenon that could become the basis for applications as diverse as microscopic robotic grippers and more efficient solar cells.

A research team at Worcester Polytechnic Institute (WPI) has developed a revolutionary, light-activated semiconductor nanocomposite material that can be used...

Im Focus: Diamonds aren't forever: Sandia, Harvard team create first quantum computer bridge

By forcefully embedding two silicon atoms in a diamond matrix, Sandia researchers have demonstrated for the first time on a single chip all the components needed to create a quantum bridge to link quantum computers together.

"People have already built small quantum computers," says Sandia researcher Ryan Camacho. "Maybe the first useful one won't be a single giant quantum computer...

Im Focus: New Products - Highlights of COMPAMED 2016

COMPAMED has become the leading international marketplace for suppliers of medical manufacturing. The trade fair, which takes place every November and is co-located to MEDICA in Dusseldorf, has been steadily growing over the past years and shows that medical technology remains a rapidly growing market.

In 2016, the joint pavilion by the IVAM Microtechnology Network, the Product Market “High-tech for Medical Devices”, will be located in Hall 8a again and will...

Im Focus: Ultra-thin ferroelectric material for next-generation electronics

'Ferroelectric' materials can switch between different states of electrical polarization in response to an external electric field. This flexibility means they show promise for many applications, for example in electronic devices and computer memory. Current ferroelectric materials are highly valued for their thermal and chemical stability and rapid electro-mechanical responses, but creating a material that is scalable down to the tiny sizes needed for technologies like silicon-based semiconductors (Si-based CMOS) has proven challenging.

Now, Hiroshi Funakubo and co-workers at the Tokyo Institute of Technology, in collaboration with researchers across Japan, have conducted experiments to...

All Focus news of the innovation-report >>>



Event News

#IC2S2: When Social Science meets Computer Science - GESIS will host the IC2S2 conference 2017

14.10.2016 | Event News

Agricultural Trade Developments and Potentials in Central Asia and the South Caucasus

14.10.2016 | Event News

World Health Summit – Day Three: A Call to Action

12.10.2016 | Event News

Latest News

Resolving the mystery of preeclampsia

21.10.2016 | Health and Medicine

Stanford researchers create new special-purpose computer that may someday save us billions

21.10.2016 | Information Technology

From ancient fossils to future cars

21.10.2016 | Materials Sciences

More VideoLinks >>>