Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

New software, HyperTools, transforms complex data into visualizable shapes

08.05.2018

Techniques provide users with insights into high-dimensional datasets

Every dataset in the observable universe has a fundamental geometry or shape to it, but that structure can be highly complicated. To make it easier to visualize complicated datasets, a Dartmouth research team has created HyperTools-- an open-source software package that leverages a suite of mathematical techniques to gain intuitions about high-dimensional datasets through the underlying geometric structures they reflect. The findings are published in the Journal of Machine Learning Research.


This is a visualization using HyperTools to represent the content of Wikipedia articles. Each dot represents a single Wikipedia article (from a set of 3,000 randomly chosen articles). The dot positions reflect what the articles are about (nearby dots are about similar topics), and the dot colors reflect automatically discovered "clusters" of articles that are about similar themes. To view a 3-D animation of this data (GIF file), go to: http://discovery.dartmouth.edu/~jmanning/hypertools_gifs/wiki.gif

Credit: Static image by Contextual Dynamics Laboratory, Dartmouth College

HyperTools can be used to transform data into visualizable shapes or animations, which can be used to: compare different datasets, gain insights into underlying patterns in an intuitive way, make generalizations across datasets, and develop and test theories relating to the Big Data.

"The datasets we're faced with as modern scientists can be enormously complex, often reflecting many interacting components," explains senior author, Jeremy R. Manning, an assistant professor of psychological and brain sciences and director of the Contextual Dynamics Lab at Dartmouth.

"Our tool turns complex data into intuitive 3-D shapes that can be visually examined and compared. Essentially, we are leveraging the visual system's amazing ability to find patterns in the world around us to also find patterns in complex scientific data."

The researchers demonstrate how HyperTools can be applied to various types of data. In the paper, they showcase visualizations of: brain activity, movie frames and brain responses to watching those frames; changes in temperature measurements across the Earth's surface from 1875 to 2013; and the thematic content of political tweets issued by Hillary Clinton and Donald Trump during the 2016 US presidential campaign.

In addition to using HyperTools to directly understand the geometric structure of data, the insights revealed by the tool can also be used to guide the development of machine learning algorithms. For example, the data visualizations can reveal how different types of observations form structured distinct clusters (e.g. Trump tweets vs. Clinton tweets) that could be used to understand the similarities and differences between groups.

As part of the HyperTools toolbox, Manning's lab continues to develop and release other types of geometric visualization analyses, including the recently launched text analyses.

###

Manning is available for comment at: jeremy.r.manning@dartmouth.edu.

The study's other authors include Dartmouth postdoctoral researcher Andrew Heusser and graduate student Kirsten Ziman (lead co-authors) and graduate student Lucy Owen, all members of Manning's lab.

GIFs and hi-res still images are available upon request.

Media Contact

Amy D. Olson
amy.d.olson@dartmouth.edu
603-646-3274

 @dartmouth

http://www.dartmouth.edu 

Amy D. Olson | EurekAlert!

More articles from Information Technology:

nachricht Touchscreens go 3D with buttons that pulsate and vibrate under your fingertips
14.03.2019 | Universität des Saarlandes

nachricht EU project CALADAN set to reduce manufacturing cost of Terabit/s capable optical transceivers
11.03.2019 | IHP - Leibniz-Institut für innovative Mikroelektronik

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Magnetic micro-boats

Nano- and microtechnology are promising candidates not only for medical applications such as drug delivery but also for the creation of little robots or flexible integrated sensors. Scientists from the Max Planck Institute for Polymer Research (MPI-P) have created magnetic microparticles, with a newly developed method, that could pave the way for building micro-motors or guiding drugs in the human body to a target, like a tumor. The preparation of such structures as well as their remote-control can be regulated using magnetic fields and therefore can find application in an array of domains.

The magnetic properties of a material control how this material responds to the presence of a magnetic field. Iron oxide is the main component of rust but also...

Im Focus: Self-healing coating made of corn starch makes small scratches disappear through heat

Due to the special arrangement of its molecules, a new coating made of corn starch is able to repair small scratches by itself through heat: The cross-linking via ring-shaped molecules makes the material mobile, so that it compensates for the scratches and these disappear again.

Superficial micro-scratches on the car body or on other high-gloss surfaces are harmless, but annoying. Especially in the luxury segment such surfaces are...

Im Focus: Stellar cartography

The Potsdam Echelle Polarimetric and Spectroscopic Instrument (PEPSI) at the Large Binocular Telescope (LBT) in Arizona released its first image of the surface magnetic field of another star. In a paper in the European journal Astronomy & Astrophysics, the PEPSI team presents a Zeeman- Doppler-Image of the surface of the magnetically active star II Pegasi.

A special technique allows astronomers to resolve the surfaces of faraway stars. Those are otherwise only seen as point sources, even in the largest telescopes...

Im Focus: Heading towards a tsunami of light

Researchers at Chalmers University of Technology and the University of Gothenburg, Sweden, have proposed a way to create a completely new source of radiation. Ultra-intense light pulses consist of the motion of a single wave and can be described as a tsunami of light. The strong wave can be used to study interactions between matter and light in a unique way. Their research is now published in the scientific journal Physical Review Letters.

"This source of radiation lets us look at reality through a new angle - it is like twisting a mirror and discovering something completely different," says...

Im Focus: Revealing the secret of the vacuum for the first time

New research group at the University of Jena combines theory and experiment to demonstrate for the first time certain physical processes in a quantum vacuum

For most people, a vacuum is an empty space. Quantum physics, on the other hand, assumes that even in this lowest-energy state, particles and antiparticles...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

International Modelica Conference with 330 visitors from 21 countries at OTH Regensburg

11.03.2019 | Event News

Selection Completed: 580 Young Scientists from 88 Countries at the Lindau Nobel Laureate Meeting

01.03.2019 | Event News

LightMAT 2019 – 3rd International Conference on Light Materials – Science and Technology

28.02.2019 | Event News

 
Latest News

To proliferate or not to proliferate

21.03.2019 | Life Sciences

Magnetic micro-boats

21.03.2019 | Physics and Astronomy

Motorless pumps and self-regulating valves made from ultrathin film

21.03.2019 | HANNOVER MESSE

VideoLinks
Science & Research
Overview of more VideoLinks >>>