Sharper AI 3D — and Linux is back! 🌙🐧
This release has been a long journey through profiling, testing and rebuilding Nightfall's AI 3D pipeline. The result is sharper depth, better performance options and the return of the Linux build.
Sharper AI 3D
The previous ZipDepth models were extremely fast, but some of the model changes needed for mobile GPU compatibility softened fine details and edges.
For this release, I profiled ZipDepth stage by stage and restored its full learned depth reconstruction while retaining the optimizations needed to run it efficiently on Quest. The new EdgePad models produce much cleaner object boundaries and preserve considerably more detail.
If you view below you can see an example of the improvement in the Depthmap Detail, as you can see that the new 256 model has better and sharper quality that the old 384 model!
AI 3D now includes:
- EdgePad-384 — the highest-quality option and the default on Quest 3/3S.
- EdgePad-256 — a faster option and the automatic Quest 2 compatibility path.
- EdgePad-224 — an experimental performance option that can approach 60 Hz depth updates on Quest 3, with some reduction in depth detail.
Auto mode selects EdgePad-384 with OpenCL on Quest 3/3S. On Quest 2 it selects EdgePad-256 with OpenGL, avoiding the unavailable OpenCL driver while retaining usable AI 3D performance.
The final depth processing has also been adjusted to preserve sharp details without reintroducing the text corruption and blocky artifacts found during testing.
I've also started working on training a custom ZipDepth model - but this will take quite a while before I have anything to show.
Better priorities for gaming
I noticed that in fast paced games, prioritizing AI 3D actually added lag to the stream, making fast paced games difficult to play, changing the processing priority fixes the issue, so I have made the AI 3D GPU Priority options clearer:
- Stream — protects video decoding and rendering performance.
- AI 3D — prioritizes maximum depth-inference throughput.
Stream is now the default. This provides noticeably better responsiveness and stream stability in fast-paced games, while AI 3D remains available for users who prefer the highest possible depth-update rate.
Linux is back
After several Android-only releases, Nightfall once again includes a Linux AppImage.
The Linux AI 3D pipeline has been rebuilt with Vulkan GPU inference. ZipDepth-384 is the default GPU model, with support for Linux systems using suitable NVIDIA, AMD or Intel Vulkan drivers.
MiDaS-256 and Depth Anything V2 remain available as CPU alternatives for compatibility and experimentation.
Something I want to talk about and I will be opening up a thread in the Discussions section, is what really is the end platform and result for the Linux version. I am really not satisfied with the double encoding via sunshine/wivrn - the quality and performance isn't great because it's streaming a full XR environment, but then at the same time you have my GPU processing zipdepth at like 1.5ms, I think some thought needs to be put into what the direction of the Linux client should be.
Additional improvements
- Added validation for GPU model tensor layouts to prevent incompatible models from reaching the inference pipeline.
- Improved Linux stream behavior and runtime refresh-rate reporting.
- Restored and documented Linux packaging and model conversion.
- Added extensive ZipDepth profiling and comparison tools for future model development.
- Continued the wider cleanup and refactoring work to make Nightfall easier and safer to maintain.