I know feelings about AI are mixed. But when AI can dream up gaussian splats in real time, from a prompt, and do refinement as you get closer to things... That's going to be pretty bonkers.

its funny that sky features like clouds and blue patches end up being fit very close to ground level because there isn't a difference in perspective to cue in the algorithm that the skybox should be tall, I wonder if there is any way to incorporate the fitting algorithm with simultaneous lidar data about true distances of things to allow scenes to be viewed from further up

I don't know anything about them but it's a cool effect. At least on this strawberry, you're not zooming in but rather traveling closer. I don't see the increasing (made up) detail you'd expect from a zoom, we sort of pop through the skin into an invented interior.

I like that they are sort of between a photo and a 3D model. Nothing quite like it.

Is there a specific name for this “dreaminess” effect?

This is like a beautiful little miniature. It's cool to see gaussian splatting applied to a detailed small thing versus a big scene.

I have a question about perhaps the most boring aspect of this strawberry: the license. you write, "You can download it under CC BY license, but attribution is appreciated rather than required." IANAL, but I'm pretty sure you can't license your work CC-BY and then waive the BY requirement in the description. Rather, you'd have to license it with something more permissive like CC0 and request attribution if you want attribution to be optional. Is that right?

Can you elaborate why you chose slang-splat over let's say Lichtfeld studio? How does it compare to the other splat-training tools?

This one keeps crashing the browser after it reaches 100%. Safari/iOS, iPhone 13. I was able to navigate and use a few of the other ones linked to from the comments, though. Curious.

I just wanted to do a quick size comparison...

If I'm reading Chrome right, that's 171 MB, for the website, and the data.

If I'm doing the math right, that's 40 seconds worth of the bandwidth from Netflix, at its highest rate.

Beautiful and quite neat to 'walk' up into the ceiling and look down. Feels like you're either quasimodo or God, depending on how nuts you are.

shame the bishop behind looks a bit flat though ;-)

Thank you for your service!

This is one of the most endearing things about open source IMHO, the way people can find novel uses for it.

There are some works on doing this directly e.g. https://arxiv.org/abs/2601.23065 but getting accurate materials is a challenge for anything more than diffuse.

AI-based relighting will no doubt start working soon.

out of the box, I imagine that surfaces can be lit, but probably not shadowed correctly. (structures aren’t solids, more like particles in 3d space)

it could look like the real-time lighting of an old game engine on rather modern assets. (quake 2-3 era)

or perhaps some "occlusion pre-pass" could be done to create a voxelized sparse volume from the splats that set a "voxel opacity value" for each to absorb light? (not far from how prebaked GI works nowadays)

note: not an expert on rendering, just a nutjob that did stuff in opengl in the old days.

It’s amazing this runs perfectly smooth on my iPhone 12 mini

That is indeed a very cool scene being about to wander around and still have decent resolution

Lots of translucent blobs composited to produce the appearance of a strawberry.

There is no mesh or model. The visual surface of the strawberry could be made up of blobs spaced far apart physically and not where the surface appears to be.

This is why they are called radiance fields, they model the light not the geometry.

Practically the blobs positions/rotations can be constrained to better physically match the geometry of a strawberry.

Others have provided details about how it works. I suggest zooming way in on that image and you'll start 'breaking though' the surface and that'll help you get an idea of how it works. Important thing is there is no defined geometric surface ("mesh"). Also important to know is that it's very, very hard to get a good splat without taking a ton of photos at different angles. It's also really, really easy to create a crappy looking splat. But when it's done right, it's a marvel

This video explains how Gaussian splatting works: https://www.youtube.com/watch?v=X8yRlA7jqEQ

https://youtube.com/watch?v=X8yRlA7jqEQ is how I learned about them. They're really cool!

Here’s a good 2 minute explainer https://youtu.be/HVv_IQKlafQ

Any idea what kind of vram is needed to run this on Nvidia?

Great video. I was about to share it here myself.

It's an artefact unfortunately. Gaussian splats have no concept of refraction and have a hard time dealing with reflection. Highlights, usually so so. Something I always wrangle.

Well, at certain angles you actually get very noticeable gaps in the strawberry that are visible when rotating it, but almost invisible when static. I think it's mostly due to that.

Thanks! We have two compressed formats, the sog by PlayCanvas and spz with sparkjs. Both now support LODs and compress really well.

Don't know if this would be in your wheelhouse, but for very nice macro splats, check out the work by Dany Bittel: https://danybittel.ch/macro.html

For example this bumblebee: https://superspl.at/scene/cf6ac78e

Edit: I completely missed that this was posted by him (:

This was also my first thought when I zoomed in into the strawberry. I wonder if you can achieve a microscope like effect with a more suitable setup. E.g. better lighting, zoom, lens, etc.

I watched this 1 month old video two days ago:

https://youtu.be/X8yRlA7jqEQ

It kinda splatted my mind.

As I understand it, Gussian Splats aren’t a method of constructing missing data in the sense you’re asking here.

But there are other, well established, methods of generating missing data in convincing manner, that are beyond my field of expertise or interest to be able to repeat here.

With the added benefit that if I’m wrong, being wrong can be a great way to spur discussion 8)

The flesh of the strawberry is very slightly translucent. By taking images from many angles, you can derive some information about the interior. This is not a separate step, it is simply the optimizer finding splats that contribute the right amount of color and light at various viewing angles. Of course those splats aren’t really going to look right to anyone who has ever eaten a strawberry, because they are only derived from the input images and not your past memories of a real strawberry.

You’ll also see the same type of problems in any incomplete part of any scene. Go poke around any of them and you’ll find places where there wasn’t enough information to make a satisfying reconstruction. For example, the back of the moon <https://superspl.at/scene/2ac8f423>, the fridge in the coffee shop <https://superspl.at/scene/6a0c3ccf>, underneath the footbridge in the forest <https://superspl.at/scene/23ebe85c> (or indeed anything in the distance), etc, etc.

It is in fact still on it's mount and slowly rotting / molding.. for a second capture :-)

Yup. It's rotten on the other side. Or maybe lead poisoned

not sure if that's a joke but i think that's just the effect of the strawberry being placed on a glass/plastic surface to be filmed from underneath

Google already has some splats on google maps,

https://youtu.be/gXug7Kb3p4I

rendering them is also non trivial from what i understand, since the millions of indivudal point positions need to be recalculated all the time while moving

This is an interesting question. There has been some attempt to model the camera better than just a pinhole camera, that could work. https://dof-gaussian.github.io/ https://github.com/leoShen917/DoF-Gaussian

My take.. at a macro scale, the dof is usually so small, that it's hard to get a reliably track. So you'd need some sort of way to tell that these stacked photos belong into a series, and then you sort of are doing focus stacking :-) I do think the alignment algorithm could be improved. Maybe the approaches I linked could be used to make a much more robust focus stacking algorithm, that also corrects for 3D geometry. That would be really cool!

If you don't focus stack and try to train on partially unfocused images, the optimizer will try to match the rendered view to be also partially unfocused.

You would have to mask out the blurry areas for each image. I guess one could just implement a feature where the optimizer only optimizes gaussians within the sharp distances relative to the camera.

It was mounted at the bottom.. and I can't quite reach it with my camera. Might have to try some two pass way.

Gaussian splat casualty. The bottom looks like partially missing from the reconstruction.

I just posted some images.. I have actually done my studio, (quality is.. let's say painterly ;-): https://superspl.at/scene/0a3916cd

Sim racing is at least kind of like that.

Works for me on Debian 13, Firefox 140.10.2esr-1~deb13u and Strix Halo gfx

Works for me with Firefox 140.10.2esr on Devuan 5.

If you can, please check any output/errors on the developer console?

And it only takes 20 Minutes to shoot all 7920 photos, the Z8 is crazy fast.

Where's the Lytro camera when you need one?

I was really pulling for OTOY to keep making light field capture and display technology...

Yes it's shoot in front of a bluescreen.

Yes mounting.. and I can't quite reach all the way from below.

I posted some pictures.. takes only 20 min!

It appears to be rendering locally, my fans started spinning up when I was wandering through another image on the site.

That doesn’t really look like Gaussian splats to me- but this video, Helicopters by A$AP Rocky, uses the technique well:

https://youtu.be/g1-46Nu3HxQ?si=y_u257DzgOg2QV83

What makes you say this is using Gaussian splats? It looks like simple motion-based cropping to me.