MIXED readers ask: How important is eye tracking for VR graphics?

Image: Midjourney is inspired by MIXED

MIXED READER Cynox wanted to know how important eye tracking will be for better visuals in VR and MR. Here is our answer.

What are MIXED readers asking?

Every Monday, we will answer a question from our readers.

Write your question in the comments or send it to the hallo on mixed.de with the subject “MIXED readers ask” and with luck we will answer it next Monday.

Question from Cynox: How important is eye tracking for high resolution, graphics and performance?

“Hello Cynox

Eye tracking has many advantages for VR and MR. It enables better looking avatars and eye tracking in social applications, can be used as a good input method as demonstrated by Apple Vision Pro, and saves a lot of energy. process with Dynamic Foveated Rendering (DFR), which can then be used. for higher resolutions and better images.

With DFR, eye tracking determines which part of the image the eye is focusing on. This allows processing power to be applied to that limited area rather than the entire field of view.

Playstation VR 2 shows the positive effect that this can have with the example of No Man’s Sky, which has seen a significant improvement in image quality due to the implementation of DFR.

Monitoring the eye on a standalone headset is very difficult to use, since such devices have strict limits of computing, power consumption, heat production, and, in the case of Meta Quest, it must be expensive it is low to produce.

Meta’s CTO has explained many times why eye tracking has not been a standard feature of Quest devices. That was the case with his last Q & A on Instagram, when he was asked what happened to the DFR Meta that spoke in 2016. Below is his answer.

“[…] To do Dynamic Foveated Rendering at the highest level, which we have been talking about for a long time as an industry, you need to follow the right monitor. The interesting thing is tracking the eye as accurately as you need to do Foveated Rendering currently takes more processing power than you get with foveated rendering. Now it is clear that it will not always be true, and there are many new ways that we can follow the eye that will make it very cheap. So in the end, this is going to be the norm, especially for mixed reality, where you want to drive high-resolution graphics without paying this quadratic tax of doing it across the entire space. visually. […]. So we haven’t given up on it, it’s still in the future, but that’s one of the reasons why it won’t happen anytime soon. ”

The Meta Quest Pro is the only Meta headset that offers eye tracking, and according to Meta, its DFR isn’t a game-changer (yet) in terms of performance gains. If eye tracking comes to one of the cheaper Quest headsets, and I think it could with the Quest 4, it probably won’t be in the form we’ve seen from the Playstation VR 2 and PC VR headsets, with IR emitters and cameras. .

Meta can rely on new eye tracking technology that is cheaper to develop and can track eye movements more accurately than previous solutions. Meta’s AR prototype Orion also has eye tracking, and AR glasses have stricter requirements than VR headsets in terms of computing, power consumption and heat output.

Startup company Inseye is currently working on an improved eye-tracking module for the Quest 3 that uses low-cost photo sensors instead of cameras for eye-tracking, providing high-resolution of sampling and lower latency than conventional monitoring. As you can see: We have not yet seen this type of technology fully developed.

Back to your question: Eye tracking is very important in the future in terms of resolution, graphics and performance, and more so with standalone headsets than with networked devices where there is less available processing power. The question is not whether eye tracking will one day become standard and enable better imaging, but in what form and at what cost.

Warm regards,

Tomislav”

Note: Links to online stores within the articles may be referred to as affiliate links. If you make a purchase through this link, MIXED receives a commission from the supplier. For you the price does not change.