Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:


New software creates dictionary for retrieving images


New software that responds to written questions by retrieving digital images has potentially broad application, ranging from helping radiologists compare mammograms to streamlining museum curators’ archiving of artwork, say the Penn State researchers who developed the technology.

Dr. James Z. Wang, assistant professor in Penn State’s School of Information Sciences and Technology and principal investigator, says the Automatic Linguistic Indexing of Pictures (ALIP) system first builds a pictorial dictionary, and then uses it for associating images with keywords. The new technology functions like a human expert who annotates or classifies terms.

"While the prototype is in its infancy, it has demonstrated great potential for use in biomedicine by reading x-rays and CT scans as well as in digital libraries, business, Web searches and the military," said Wang, who holds the PNC Technologies Career Development Professorship at IST and also is a member of the Department of Computer Science and Engineering.

ALIP processes images the way people seem to. When we see a new kind of vehicle with two wheels, a seat and a handlebar, for instance, we recognize it as "a bicycle" from information about related images stored in our brains. ALIP has a similar bank of statistical models "learned" from analyzing image features.

The system is detailed in a paper, "Learning-based Linguistic Indexing of Pictures with 2-D MHMMs," to be given today (Dec. 4) at the Association of Computing Machinery’s (ACM) Multimedia Conference in Juan Les Pins, France. Co-author is Dr. Jia Li, Penn State assistant professor of statistics.

Unlike other content-based retrieval systems that compare features of visually similar images, ALIP uses verbal cues that range from simple concepts such as "flowers" and "mushrooms" to higher-level ones such as "rural" and "European." ALIP also can classify images into a larger number of categories than other systems, thereby broadening the uses of image databases.

Other advantages include ALIP’s abilities to be trained with a relatively large number of concepts simultaneously and with images that are not necessarily visually similar.

In one experiment, Wang and Li "trained" ALIP with 24,000 photographs found on 600 CD-ROMs, with each CD-ROM collection assigned keywords to describe its content. After "learning" these images, the computer then automatically created a dictionary of concepts such as "building," "landscape," and "European." Statistical modeling enabled ALIP to automatically index new or unlearned images with the linguistic terms of the dictionary.

Wang tested that dictionary with 5,000 randomly selected images to see if the computer could provide meaningful keyword annotations for the new images. His conclusion: The more specific the query for an image, the higher the system’s degree of accuracy in retrieving an appropriate image.

Wang and Li are using ALIP as part of a three-year National Science Foundation research project to develop digital imagery technologies for the preservation and cataloguing of Asian art and cultural heritages. This research aims to bypass or reduce the efforts in the labor-intensive creation and entry of manual descriptions or artwork.

Eventually, the system is expected to identify the discriminating features of Chinese landscape paintings and the distinguishing characteristics of paintings from different historical periods, Wang notes.

The researchers’ progress in the first year of that project is discussed in the paper, "Interdisciplinary Research to Advance Digital Imagery Indexing and Retrieval Technologies for Asian Art and Cultural Heritages." The research will be presented on Dec. 6 at in a special session of ACM’s Multimedia Conference in France.

Further research will be aimed at improving ALIP’s accuracy and speed.

ALIP’s reading of a beach scene with sailboats yielded the keyword annotations of "ocean," "paradise," "San Diego," "Thailand," "beach" and "fish." Even though the computer was intelligent enough to recognize the high-level concept of "paradise," additional research will focus on making the technology more accurate, so that San Diego and Thailand will not appear in the annotation of the same picture, Wang says.

"This system has the potential to change how we handle images in our daily life by giving us better and more access," Wang says. Wang and Li’s latest research builds on their earlier efforts at Stanford University. Sun Microsystems provided most of the equipment used in the project.

Margaret Hopkins | EurekAlert!
Further information:

More articles from Information Technology:

nachricht Green Light for Galaxy Europe
15.03.2018 | Albert-Ludwigs-Universität Freiburg im Breisgau

nachricht Tokyo Tech's six-legged robots get closer to nature
12.03.2018 | Tokyo Institute of Technology

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Locomotion control with photopigments

Researchers from Göttingen University discover additional function of opsins

Animal photoreceptors capture light with photopigments. Researchers from the University of Göttingen have now discovered that these photopigments fulfill an...

Im Focus: Surveying the Arctic: Tracking down carbon particles

Researchers embark on aerial campaign over Northeast Greenland

On 15 March, the AWI research aeroplane Polar 5 will depart for Greenland. Concentrating on the furthest northeast region of the island, an international team...

Im Focus: Unique Insights into the Antarctic Ice Shelf System

Data collected on ocean-ice interactions in the little-researched regions of the far south

The world’s second-largest ice shelf was the destination for a Polarstern expedition that ended in Punta Arenas, Chile on 14th March 2018. Oceanographers from...

Im Focus: ILA 2018: Laser alternative to hexavalent chromium coating

At the 2018 ILA Berlin Air Show from April 25–29, the Fraunhofer Institute for Laser Technology ILT is showcasing extreme high-speed Laser Material Deposition (EHLA): A video documents how for metal components that are highly loaded, EHLA has already proved itself as an alternative to hard chrome plating, which is now allowed only under special conditions.

When the EU restricted the use of hexavalent chromium compounds to special applications requiring authorization, the move prompted a rethink in the surface...

Im Focus: Radar for navigation support from autonomous flying drones

At the ILA Berlin, hall 4, booth 202, Fraunhofer FHR will present two radar sensors for navigation support of drones. The sensors are valuable components in the implementation of autonomous flying drones: they function as obstacle detectors to prevent collisions. Radar sensors also operate reliably in restricted visibility, e.g. in foggy or dusty conditions. Due to their ability to measure distances with high precision, the radar sensors can also be used as altimeters when other sources of information such as barometers or GPS are not available or cannot operate optimally.

Drones play an increasingly important role in the area of logistics and services. Well-known logistic companies place great hope in these compact, aerial...

All Focus news of the innovation-report >>>



Industry & Economy
Event News

Ultrafast Wireless and Chip Design at the DATE Conference in Dresden

16.03.2018 | Event News

International Tinnitus Conference of the Tinnitus Research Initiative in Regensburg

13.03.2018 | Event News

International Virtual Reality Conference “IEEE VR 2018” comes to Reutlingen, Germany

08.03.2018 | Event News

Latest News

Wandering greenhouse gas

16.03.2018 | Earth Sciences

'Frequency combs' ID chemicals within the mid-infrared spectral region

16.03.2018 | Physics and Astronomy

Biologists unravel another mystery of what makes DNA go 'loopy'

16.03.2018 | Life Sciences

Science & Research
Overview of more VideoLinks >>>