Memory is a wonderfully useful yet strangely mysterious thing, without which we would never learn from our mistakes, recognize our friends, or find our way around the world. Exactly what makes certain things memorable or forgetful, however, is a baffling concept that a team of scientists at the Massachusetts Institute of Technology (MIT) have been working hard to decipher.
After conducting extensive research, the team has now created a convolutional neural network (CNN) with the ability to accurately predict the "memorability" of photographs, thereby shedding light on how human memory works.
You can try this out yourself with an online app that tells you how memorable your photographs are. It produces a heat map that indicates which elements of these images are the most memorable or forgettable.
CNNs are artificial networks of neurons designed in accordance with the arrangement of neural cells in the visual cortex – the part of the brain that processes visual information. These networks are capable of deep learning, which involves processing large volumes of data in order to identify underlying patterns. In other words, they learn information for themselves instead of requiring pre-programming.
Publishing their study via MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), the researchers explain how they first conducted a series of tests in order to determine how humans memorize photographs. This involved displaying a stream of images, some of which were repeated, and asking participants to press a button each time they recognized a photograph that they had already seen.
Analyzing the data obtained during these tests, the team found a rank correlation (a measure of comparing two sets of data) of 0.68 between the human subjects' responses and the actual rate of image repetition.
Taking this a step further, the researchers sought to identify which features of a photograph are responsible for its memorability. For instance, they found that pictures of people were generally more memorable than natural landscapes.
They then created an algorithm to predict how memorable or forgettable images are, and found that their CNN was able to achieve a rank correlation of 0.64. The fact that this score is so close to that obtained by the human subjects suggests that the "MemNet" algorithm is an accurate model for predicting the memorability of images.
In their study, the researchers explained that this could have a wide range of real-world implications. For example, understanding what makes things memorable may enable the manipulation of data in order to increase its memorability, thereby ensuring important facts are not forgotten.
In a statement, study coauthor Aditya Khosla claimed that “we could potentially improve people’s memory if we present them with memorable images.”
Image in text: For each image, the MemNet algorithm creates a heat map identifying its most memorable and forgettable regions. Credit: MIT's Computer Science and Artificial Intelligence Lab
