This week, I’ve been messing around with a photogrammetry and rendering technique that might’ve redlined my brain. Hat tip to Jonathan Stephens for putting this old technique turned anew on my radar.

3D Gaussian Splats

Okay, what the heck is a gaussian splat? It’s helpful to start with a photogrammetric technique most of us are familiar with, Structure from Motion (SfM). That’s the algorithm programs like Pix4D use to calculate point position from a bunch of photos captured by a drone.

Now, instead of representing all the measurements as points or polygons, they’re represented by gaussians (ellipsoids). These gaussians can change shape, size, position, color, as well as transparency, and are ultimately trained via the original photoset to take on the best representative form. In addition, they adapt depending on perspective, offering next-level rendering at next-level speed. The paper that started the recent madness is here, if you’re interested in a deep dive, and the video below does a great job explaining the concept.

 
 

​I think there’s a lot of potential for a technology like this in recon: renderings of collision scenes, vehicles, quick and beautiful modeling, etc. As such, I’ve been experimenting with two freely available web-based G-Splat tools, Luma and Polycam.

When exporting in those formats, I’m really just capitalizing on the implementation of the SfM algorithm and cloud processing, as the rendering benefits are only realized in the native viewer or in something like Unreal Engine, for now. The helmet point cloud below was created using a 50-second iPhone video and Luma. Notice that the face shield, which was clear, is well represented. That’s rare for a photogrammetric technique, and in processing the same video using RealityCapture (a more traditional photogrammetry tool), that data was missing.

 
 
 
 

Much more to learn, but hopefully this serves as a good intro to the technology.

Have a great weekend!

Lou Peck
​Lightpoint | Axiom​

P.S. My photogrammetry course was launched this week to those that expressed an interest and one seat remains. Registration will be available to the gen pop early next week, so please sign up before Monday if you'd like to attend. Hope to see you there!