People intuitively find good solutions even in complex, unfamiliar situations. They learn quickly and are able to cope with an almost infinite number of options. A team of researchers have created a game to experimentally investigate how this is possible and found astonishing parallels to state-of-the-art machine algorithms. The results of the study have been published in Nature Human Behaviour.
Which career is right for me? Which pension scheme should I invest in? Which chess move should I make next? In many of the decisions we make, the number of possible actions is vast. Theoretically, we can only find out which option is best for us by trying out all the possibilities.
As that is clearly impossible, we have to either rely on tried-and-tested options or take risks in trying out new ones. In all our decisions, we tread an intuitive balance between these two poles — and often manage to find good solutions. The dilemma between exploring new opportunities and sticking to what we know is termed the “exploration-exploitation dilemma.”
It is of interest not only to psychologists, but also to computer scientists who program learning algorithms. They need to make sure that the algorithm balances two tasks: drawing on information that is known while also exploring new and unknown options — for example, when making personalized recommendations for customers in online shops.
But how exactly do people navigate situations with only a few known options against a vast set of unknown ones? And can machine learning algorithms handle these same computational challenges?
To investigate these questions, a team of researchers from the Max Planck Institute for Human Development, Harvard University, University College London, and the University of Surrey developed a series of experimental decision games. The games were played by 241 people, as well as by computers applying different models and algorithms.
In the online games, human and virtual players were presented with a grid of tiles. Each tile was worth a certain number of points; this reward was revealed only when the tile was clicked. Because the number of clicks available to each player was considerably smaller than the number of tiles, the players could only try out a limited number of options.
Initially, neither the humans nor the computers knew whether there was any pattern in the spatial arrangement of tiles in terms of whether neighboring tiles had similar point values. With time, they became aware of the spatial patterns behind the tiles and used this knowledge to develop a strategy that balanced clicking on neighboring tiles (exploitation) and opting for riskier, more distant tiles (exploration).
“People don’t have to experience every option in order to make good decisions. They are able to generalize from the basis of just a few known options and make predictions about where exploration may seem promising,” says Charley Wu, lead author and doctoral student at the Max Planck Institute for Human Development. “We also found that people are quite optimistic and curious, actively exploring the most unfamiliar options.”
The learning algorithms were also fairly successful — some more than others. In a direct comparison of human behavior and algorithms, the researchers found that combining two common algorithms enabled them to predict human behavior surprisingly well. The first algorithm predicted the spatial structure of the environment by generalization; the second simulated “curiosity” and tended to try out unknown tiles.
“The results of our study show how machine learning methods can help us gain a better understanding of human decision making. Conversely, psychological studies can help to improve the algorithms used in computer science — because in most areas human intelligence is still far superior to machine intelligence,” says Björn Meder, adjunct researcher at the Max Planck Institute for Human Development.
The Max Planck Institute for Human Development in Berlin was founded in 1963. It is an interdisciplinary research institution dedicated to the study of human development and education. The Institute belongs to the Max Planck Society for the Advancement of Science, one of the leading organizations for basic research in Europe.
Wu, C. M., Schulz, E., Speekenbrink, M., Nelson, J.D., & Meder, B. (2018). Generalization guides human exploration in vast decision spaces. Nature Human Behaviour. doi.org/10.1038/s41562-018-0467-4
Kerstin Skork | Max-Planck-Institut für Bildungsforschung
A gene activated in infant and young brains determines learning capacity in adulthood
13.11.2018 | Universitätsklinikum Hamburg-Eppendorf
The Maturation Pattern of the Hippocampus Drives Human Memory Deve
23.07.2018 | Max-Planck-Institut für Bildungsforschung
DESY and MPSD scientists create high-order harmonics from solids with controlled polarization states, taking advantage of both crystal symmetry and attosecond electronic dynamics. The newly demonstrated technique might find intriguing applications in petahertz electronics and for spectroscopic studies of novel quantum materials.
The nonlinear process of high-order harmonic generation (HHG) in gases is one of the cornerstones of attosecond science (an attosecond is a billionth of a...
Nano- and microtechnology are promising candidates not only for medical applications such as drug delivery but also for the creation of little robots or flexible integrated sensors. Scientists from the Max Planck Institute for Polymer Research (MPI-P) have created magnetic microparticles, with a newly developed method, that could pave the way for building micro-motors or guiding drugs in the human body to a target, like a tumor. The preparation of such structures as well as their remote-control can be regulated using magnetic fields and therefore can find application in an array of domains.
The magnetic properties of a material control how this material responds to the presence of a magnetic field. Iron oxide is the main component of rust but also...
Due to the special arrangement of its molecules, a new coating made of corn starch is able to repair small scratches by itself through heat: The cross-linking via ring-shaped molecules makes the material mobile, so that it compensates for the scratches and these disappear again.
Superficial micro-scratches on the car body or on other high-gloss surfaces are harmless, but annoying. Especially in the luxury segment such surfaces are...
The Potsdam Echelle Polarimetric and Spectroscopic Instrument (PEPSI) at the Large Binocular Telescope (LBT) in Arizona released its first image of the surface magnetic field of another star. In a paper in the European journal Astronomy & Astrophysics, the PEPSI team presents a Zeeman- Doppler-Image of the surface of the magnetically active star II Pegasi.
A special technique allows astronomers to resolve the surfaces of faraway stars. Those are otherwise only seen as point sources, even in the largest telescopes...
Researchers at Chalmers University of Technology and the University of Gothenburg, Sweden, have proposed a way to create a completely new source of radiation. Ultra-intense light pulses consist of the motion of a single wave and can be described as a tsunami of light. The strong wave can be used to study interactions between matter and light in a unique way. Their research is now published in the scientific journal Physical Review Letters.
"This source of radiation lets us look at reality through a new angle - it is like twisting a mirror and discovering something completely different," says...
11.03.2019 | Event News
01.03.2019 | Event News
28.02.2019 | Event News
22.03.2019 | Life Sciences
22.03.2019 | Life Sciences
22.03.2019 | Information Technology