Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

New software, HyperTools, transforms complex data into visualizable shapes

08.05.2018

Techniques provide users with insights into high-dimensional datasets

Every dataset in the observable universe has a fundamental geometry or shape to it, but that structure can be highly complicated. To make it easier to visualize complicated datasets, a Dartmouth research team has created HyperTools-- an open-source software package that leverages a suite of mathematical techniques to gain intuitions about high-dimensional datasets through the underlying geometric structures they reflect. The findings are published in the Journal of Machine Learning Research.


This is a visualization using HyperTools to represent the content of Wikipedia articles. Each dot represents a single Wikipedia article (from a set of 3,000 randomly chosen articles). The dot positions reflect what the articles are about (nearby dots are about similar topics), and the dot colors reflect automatically discovered "clusters" of articles that are about similar themes. To view a 3-D animation of this data (GIF file), go to: http://discovery.dartmouth.edu/~jmanning/hypertools_gifs/wiki.gif

Credit: Static image by Contextual Dynamics Laboratory, Dartmouth College

HyperTools can be used to transform data into visualizable shapes or animations, which can be used to: compare different datasets, gain insights into underlying patterns in an intuitive way, make generalizations across datasets, and develop and test theories relating to the Big Data.

"The datasets we're faced with as modern scientists can be enormously complex, often reflecting many interacting components," explains senior author, Jeremy R. Manning, an assistant professor of psychological and brain sciences and director of the Contextual Dynamics Lab at Dartmouth.

"Our tool turns complex data into intuitive 3-D shapes that can be visually examined and compared. Essentially, we are leveraging the visual system's amazing ability to find patterns in the world around us to also find patterns in complex scientific data."

The researchers demonstrate how HyperTools can be applied to various types of data. In the paper, they showcase visualizations of: brain activity, movie frames and brain responses to watching those frames; changes in temperature measurements across the Earth's surface from 1875 to 2013; and the thematic content of political tweets issued by Hillary Clinton and Donald Trump during the 2016 US presidential campaign.

In addition to using HyperTools to directly understand the geometric structure of data, the insights revealed by the tool can also be used to guide the development of machine learning algorithms. For example, the data visualizations can reveal how different types of observations form structured distinct clusters (e.g. Trump tweets vs. Clinton tweets) that could be used to understand the similarities and differences between groups.

As part of the HyperTools toolbox, Manning's lab continues to develop and release other types of geometric visualization analyses, including the recently launched text analyses.

###

Manning is available for comment at: jeremy.r.manning@dartmouth.edu.

The study's other authors include Dartmouth postdoctoral researcher Andrew Heusser and graduate student Kirsten Ziman (lead co-authors) and graduate student Lucy Owen, all members of Manning's lab.

GIFs and hi-res still images are available upon request.

Media Contact

Amy D. Olson
amy.d.olson@dartmouth.edu
603-646-3274

 @dartmouth

http://www.dartmouth.edu 

Amy D. Olson | EurekAlert!

More articles from Information Technology:

nachricht FaceHaptics – Simulation for all senses in VR
02.04.2020 | Hochschule Bonn-Rhein-Sieg

nachricht Pollen measurement system developed at TU Graz analyses pollen fast, cheaply and automatically
02.04.2020 | Technische Universität Graz

All articles from Information Technology >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: A sensational discovery: Traces of rainforests in West Antarctica

90 million-year-old forest soil provides unexpected evidence for exceptionally warm climate near the South Pole in the Cretaceous

An international team of researchers led by geoscientists from the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research (AWI) have now...

Im Focus: Blocking the Iron Transport Could Stop Tuberculosis

The bacteria that cause tuberculosis need iron to survive. Researchers at the University of Zurich have now solved the first detailed structure of the transport protein responsible for the iron supply. When the iron transport into the bacteria is inhibited, the pathogen can no longer grow. This opens novel ways to develop targeted tuberculosis drugs.

One of the most devastating pathogens that lives inside human cells is Mycobacterium tuberculosis, the bacillus that causes tuberculosis. According to the...

Im Focus: Physicist from Hannover Develops New Photon Source for Tap-proof Communication

An international team with the participation of Prof. Dr. Michael Kues from the Cluster of Excellence PhoenixD at Leibniz University Hannover has developed a new method for generating quantum-entangled photons in a spectral range of light that was previously inaccessible. The discovery can make the encryption of satellite-based communications much more secure in the future.

A 15-member research team from the UK, Germany and Japan has developed a new method for generating and detecting quantum-entangled photons at a wavelength of...

Im Focus: Junior scientists at the University of Rostock invent a funnel for light

Together with their colleagues from the University of Würzburg, physicists from the group of Professor Alexander Szameit at the University of Rostock have devised a “funnel” for photons. Their discovery was recently published in the renowned journal Science and holds great promise for novel ultra-sensitive detectors as well as innovative applications in telecommunications and information processing.

The quantum-optical properties of light and its interaction with matter has fascinated the Rostock professor Alexander Szameit since College.

Im Focus: Stem Cells and Nerves Interact in Tissue Regeneration and Cancer Progression

Researchers at the University of Zurich show that different stem cell populations are innervated in distinct ways. Innervation may therefore be crucial for proper tissue regeneration. They also demonstrate that cancer stem cells likewise establish contacts with nerves. Targeting tumour innervation could thus lead to new cancer therapies.

Stem cells can generate a variety of specific tissues and are increasingly used for clinical applications such as the replacement of bone or cartilage....

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

13th AKL – International Laser Technology Congress: May 4–6, 2022 in Aachen – Laser Technology Live already this year!

02.04.2020 | Event News

“4th Hybrid Materials and Structures 2020” takes place over the internet

26.03.2020 | Event News

Most significant international Learning Analytics conference will take place – fully online

23.03.2020 | Event News

 
Latest News

Most of Earth's carbon was hidden in the core during its formative years

02.04.2020 | Earth Sciences

Discovery of life in solid rock deep beneath sea may inspire new search for life on Mars

02.04.2020 | Life Sciences

Geneticists are bringing personal medicine closer to recently admixed individuals

02.04.2020 | Health and Medicine

VideoLinks
Science & Research
Overview of more VideoLinks >>>