Author here, I'm glad to have finally figured out why a bunch of people followed me on Twitter this morning!
If you enjoyed this paper, there's a companion blog post about the actual process of writing it: https://www.mattkeeter.com/projects/siggraph/
(and I'm happy to answer questions, of course)
Super cool to see Matthew's library as well: https://libfive.com/studio/
Fabulous work. The video presentation is only 18 minutes long, well organized, and very accessible -- the author does a great job of explaining how and why the rendering works so efficiently using a simple example in 2D:
https://www.youtube.com/watch?v=_6CnaugAcCc
Highly recommended.
Really cool!
Need neural radiance fields. Then add super resolution, then add motion prediction and you are on your way to a synthetic visual cortex.
In the future GPUs will be spec'd by how far in the future they can predict given a power and thermal envelope.
Very very impressive work. I wonder if it would be feasible for vendors to implement the interpreter in hardware in future.
this isn't exactly the right place to ask but i'm betting people interested in graphics pipeline latency will visit this comment section:
how do people that absolutely need the lowest latency numbers make do with GPUs? i'm not in graphics but i'm in ML and lately i've been working on research to squeeze as much juice out GPUs as possible. my last project involved optimizing a pipeline that basically consisted of just a stack of filters and some min/max finding. the fastest i could get it to go after throwing everything i could at it and i only got it down to ~20ms. that's ~50Hz. admittedly it was a tall stack but still i don't understand how game devs (for example) get complex pipelines to finish within the 60Hz/16ms given you don't never have access to bare metal GPU.
Looking forward to watching this when I have some down time... But, whew, I did a double-take when I saw "VR/ML workstation"
This is important for vision ML because sufficiently advanced simulations may comprise the real world as a special case: https://arxiv.org/abs/1703.06907
For example, gaming pixel maps have been used for semantic simulation, and have been rendered scenes: https://arxiv.org/abs/1608.02192 https://www.cv-foundation.org/openaccess/content_cvpr_2016/p...
This concept has not (yet) been applied in audio ML. We have a paper in submission---will be on ArXiv soon---where we share a GPU-enabled modular synthesizer that is 16000x faster than realtime, concurrently released with a 1-billion audio sample corpus that is 100x larger than any audio dataset in the literature. Here's the code: https://github.com/torchsynth/torchsynth