The system works by taking advantage of the multiple sensors on a mobile phone, as well as those of other mobile phones in the vicinity.
Dubbed TagSense, the new app was developed by students from Duke University and the University of South Carolina (USC) and unveiled at the ninth Association for Computing Machinery's International Conference on Mobile Systems, Applications and Services (MobiSys), being held in Washington, D.C.
"In our system, when you take a picture with a phone, at the same time it senses the people and the context by gathering information from all the other phones in the area," said Xuan Bao, a Ph.D. student in computer science at Duke who received his master's degree at Duke in electrical and computer engineering.
Bao and Chuan Qin, a visiting graduate student from USC, developed the app working with Romit Roy Choudhury, assistant professor of electrical and computer engineering at Duke's Pratt School of Engineering. Qin and Bao are currently involved in summer internships at Microsoft Research.
"Phones have many different kinds of sensors that you can take advantage of," Qin said. "They collect diverse information like sound, movement, location and light. By putting all that information together, you can sense the setting of a photograph and describe its attributes."
By using information about the environment of a photograph, the students believe they can achieve a more accurate tagging of a particular photograph than could be achieved by facial recognition alone. Such information about a photograph's entirety provides additional details that can then be searched at a later time.
For example, the phone's built-in accelerometer can tell if a person is standing still for a posed photograph, bowling or even dancing. Light sensors in the phone's camera can tell if the shot is being taken indoors or outdoors on a sunny or cloudy day. The sensors can also approximate environmental conditions – such as snow or rain -- by looking up the weather conditions at that time and location. The microphone can detect whether or not a person in the photograph is laughing, or quiet. All of these attributes are then assigned to each photograph, the students said.
Bao pointed out that with multiple tags describing more than just a particular person's name, it would be easier to not only organize an album of photographs for future reference, but find particular photographs years later. With the exploding number of digital pictures in the cloud and in our personal computers, the ability to easily search and retrieve desired pictures will be valuable in the future, he said.
"So, for example, if you've taken a bunch of photographs at a party, it would be easy at a later date to search for just photographs of happy people dancing," Qin said. "Or more specifically, what if you just wanted to find photographs only of Mary dancing at the party and didn't want to look through all the photographs of Mary?"
These added details of automatic tagging could help complement existing tagging applications, according to senior researcher Roy Choudhury.
"While facial recognition programs continue to improve, we believe that the ability to identify photographs based on the setting of the photograph can lead to a richer, more detailed way to tag photographs," Roy Choudhury said. "TagSense was compared to Apple's iPhoto and Google's Picasa, and showed that it can provide greater sophistication in tagging photographs."
The students envision that TagSense would most likely be adopted by groups of people, such as friends, who would "opt in," allowing their mobile phone capabilities to be harnessed when members of the group were together. Importantly, Roy Choudhury added, TagSense would not request sensed data from nearby phones that do not belong to this group, thereby protecting users' privacy.
The experiments were conducted using eight Google Nexus One mobile phones on more than 200 photos taken at various locations across the Duke campus, including classroom buildings, gyms and the art museum.
The current application is a prototype, and the researchers believe that a commercial product could be available in a few years.
Srihari Nelakuditi, associate professor of computer science and engineering at USC, was also a member of the research team. The research is supported by the National Science Foundation. Roy Choudhury's Systems Networking Research Group is also supported by Microsoft, Nokia, Verizon, and Cisco.
Richard Merritt | EurekAlert!
Next Generation Cryptography
20.03.2018 | Fraunhofer-Institut für Sichere Informationstechnologie SIT
TIB’s Visual Analytics Research Group to develop methods for person detection and visualisation
19.03.2018 | Technische Informationsbibliothek (TIB)
Satellites in near-Earth orbit are at risk due to the steady increase in space debris. But their mission in the areas of telecommunications, navigation or weather forecasts is essential for society. Fraunhofer FHR therefore develops radar-based systems which allow the detection, tracking and cataloging of even the smallest particles of debris. Satellite operators who have access to our data are in a better position to plan evasive maneuvers and prevent destructive collisions. From April, 25-29 2018, Fraunhofer FHR and its partners will exhibit the complementary radar systems TIRA and GESTRA as well as the latest radar techniques for space observation across three stands at the ILA Berlin.
The "traffic situation" in space is very tense: the Earth is currently being orbited not only by countless satellites but also by a large volume of space...
An international team of researchers has discovered a new anti-cancer protein. The protein, called LHPP, prevents the uncontrolled proliferation of cancer cells in the liver. The researchers led by Prof. Michael N. Hall from the Biozentrum, University of Basel, report in “Nature” that LHPP can also serve as a biomarker for the diagnosis and prognosis of liver cancer.
The incidence of liver cancer, also known as hepatocellular carcinoma, is steadily increasing. In the last twenty years, the number of cases has almost doubled...
In just a few weeks from now, the Chinese space station Tiangong-1 will re-enter the Earth's atmosphere where it will to a large extent burn up. It is possible that some debris will reach the Earth's surface. Tiangong-1 is orbiting the Earth uncontrolled at a speed of approx. 29,000 km/h.Currently the prognosis relating to the time of impact currently lies within a window of several days. The scientists at Fraunhofer FHR have already been monitoring Tiangong-1 for a number of weeks with their TIRA system, one of the most powerful space observation radars in the world, with a view to supporting the German Space Situational Awareness Center and the ESA with their re-entry forecasts.
Following the loss of radio contact with Tiangong-1 in 2016 and due to the low orbital height, it is now inevitable that the Chinese space station will...
Fraunhofer Institute for Organic Electronics, Electron Beam and Plasma Technology FEP, provider of research and development services for OLED lighting solutions, announces the founding of the “OLED Licht Forum” and presents latest OLED design and lighting solutions during light+building, from March 18th – 23rd, 2018 in Frankfurt a.M./Germany, at booth no. F91 in Hall 4.0.
They are united in their passion for OLED (organic light emitting diodes) lighting with all of its unique facets and application possibilities. Thus experts in...
A new scenario seeking to explain how Mars' putative oceans came and went over the last 4 billion years implies that the oceans formed several hundred million...
23.03.2018 | Event News
19.03.2018 | Event News
16.03.2018 | Event News
23.03.2018 | Materials Sciences
23.03.2018 | Agricultural and Forestry Science
23.03.2018 | Physics and Astronomy