Modeling Human Movement Perception with a Computer Algorithm

Post by Anastasia Sares

What's the science?

Picture a person dancing. How can you recognize all of those body parts as belonging to one individual, even when they aren’t all moving together? Our brains have an ability to track and predict complex movement, and this could be because we use a hierarchical structure to organize everything. In a hierarchical structure, the hand has its own movements, but it also inherits the motion of the arm, which in turn inherits the motion of the torso, and all of these movements are added up to produce the motion we see. If we are aware of these relationships, we can track and predict the movement more accurately. This week in PNAS, Bill and colleagues showed that giving hierarchical information to a movement-tracking algorithm allowed it to mimic human performance levels.

How did they do it?

Modeling is an increasingly popular approach that differs from classic experimental science. The logic goes like this: if we can construct a computer algorithm that successfully mimics something in the real world (human movement perception, for example), then the characteristics of that computer algorithm might help us understand how the real process works.

The authors first brought human participants into the lab to test their motion perception. The participants had to track a few target dots moving around a circle while ignoring other distracter dots. In some conditions, the moving dots moved independently, and in other conditions they had a hidden hierarchical structure (You can find video examples of the task here). The humans performed significantly better when there was a hierarchical structure.

After the data from the human participants were collected, the authors created a computer algorithm (a Bayesian observer model)  that had human-like performance only on the non-hierarchical stimuli. They then tested what would happen before and after they improved the model by giving it access to hierarchical structures.

What did they find?

At first, when the algorithm was exposed to the hierarchical condition, it did not improve as the humans had, indicating that it did not have enough information. However, when it had access to hierarchical information, the algorithm’s performance resembled human data much more closely. This suggests that the addition of hierarchical information was the missing element necessary for better motion tracking.

What's the impact?

Getting a computer algorithm to reproduce human behavior doesn’t prove that the human and the algorithm work the same way, but it does give us an idea of the minimum amount of information necessary, and a plausible mechanism for how it might work. The next step would be to actually find this mechanism at work in the human brain.

Bill et al. Hierarchical structure is employed by humans during visual motion perception. Proceedings of the National Academy of Sciences (2020). Access the original scientific publication here.