Random Gazes, Telling Eyes

May 28th, 2025

Jonas De Bruyne

You’re watching someone put together an IKEA cupboard. Thrilling, isn’t it? The constant struggle, the rising frustration, the small victories, and then the final blow before giving up… or better yet, the wave of relief when it finally comes together. Lessons learned? It's not just a cupboard—it’s a test of character.

There’s something about this situation, however, that is even more fascinating: you, as a human observer, have a knack for interpreting the body language cues—some more subtle than others—and recognizing and categorizing the different emotional and cognitive states. For example, when someone frowns while looking at the instructions, it’s a clear sign of confusion—or indigestion. Either way, it’s not great.

Now, imagine if we could evaluate IKEA instructions simply by adding a sensor to the person assembling the furniture—automatically detecting moments of hesitation, uncovering points of confusion, and using that insight to redesign the instructions for a smoother experience. It is precisely this kind of potential that our research last year so clearly revealed—yes, I delayed writing this bog post, I must admit.

Of course, it’s not quite that simple. Strapping a sensor to one person and expecting instant insight would be nice, but we’re not there yet. Human behavior is messy—someone might pause because they’re confused by the instructions, or because they suddenly realized their life choices led them to this moment. That’s why, in our study, we leaned on the power of numbers. A single blip in the data might be noise. But when the algorithm flags behavior at step 9 for everyone, it’s probably not coincidence. It’s a clue. And it’s those collective patterns that point us toward better design of the instruction of step 9.

So here’s what we did. Together with Howest - HIT Lab and WAAK, we set up a study to see what eye tracking could tell us during cable assembly tasks. (Yes, we moved on from IKEA cupboards to cables—equally thrilling, but with more wires and fewer missing screws.) To give you a bit of context: the cable assembly board at hand is already being used to assess novice workers’ skills and abilities. We wanted to see if adding eye tracking could bring in another, more objective layer of insight.

But we didn’t stop there. We also used the study to test something a bit more experimental: whether a concept called gaze transition entropy (GTE)—basically, how randomly someone’s eyes bounce around between (predefined) areas—could serve as a reliable indicator of hesitation. If it works, it could eventually become part of a broader toolkit to evaluate and improve instructional designs—to identify the notorious step 9 and improve its instructions.

Conceptual image of gaze transition entropy. Left: low gaze transition entropy, fully predictable gaze path; Right: higher gaze transition entropy, less predictable gaze path

So what did we actually find? In short: there’s promise. When we looked at the gaze patterns—specifically, GTE—we noticed something interesting. The more experienced the participants became, the more structured and predictable their eye movements got. Less bouncing around, a more decisive gaze path. In other words, lower GTE.

That’s encouraging, because it suggests that GTE might work as an objective marker for proficiency. But what really caught our attention was a subtle trend: when participants reported feeling hesitant during certain steps, their GTE was a bit higher. Their eyes darted around more unpredictably, reflecting the mental scramble we all know when instructions aren’t quite making sense.

Yes—but what about irrelevant noisy gaze patterns, like because people are reading, you ask? We explored a variation of the measure (modified GTE) that filters out shifting gazes within the same areas (like when you’re reading a text presented within the area instruction manual). Both versions seemed to capture something meaningful, especially when viewed across the whole group.

Now, is GTE a magic number that will instantly tell us which instruction needs fixing? Not quite. But it is a promising candidate to become part of an evaluation toolkit. When lots of people get stuck on the same step, and the data backs it up with chaotic gaze patterns, that’s a strong case for redesigning that part of the process.

And while this was just a first step—an exploratory study with a limited sample—the potential is clear. GTE opens up new possibilities for evaluating instructional design and at the same time can inspire researchers in other fields. Whether it's in training, education, cockpit design, healthcare, or even human–robot interaction, the ability to objectively detect moments of hesitation or uncertainty through gaze patterns can become a valuable tool. With further development, this line of work can certainly help shape future evaluation practices across domains—moving from subjective impressions to data-driven insights based on actual user behavior.

Even more exciting is the potential for real-time applications. Imagine adaptive systems that respond instantly to signs of confusion: updating instructions, offering support, or adjusting difficulty on the fly. GTE could be a key building block in creating more responsive, personalized, and effective user experiences.

As such, this work has shed some interesting light on future possibilities. For now, though, we’ll take the small victory—kind of like having assembled that IKEA cupboard with no leftover parts. A cupboard that appeared useful to showcase our little trophy: with this work, we won the Best Paper Award in the track of Industrial Cognitive Ergonomics and Engineering Psychology at AHFE2024 (International Conference on Applied Human Factors and Ergonomics).

For those interested in the full work and technical details, the paper is available here.