Well if you take video or many photos of a house you can create a 3D model using photogrammetry. I was recently served an ad showing that I think Redfin includes 3D photogrammetry captures of some homes. So it’s perhaps already being done in the wild.
There are a number of companies operating in the space. Matterport is one some people in my work have been exploring for documenting projects in various stages of construction. There are definitely still a bunch of rough edges, but the results so far even just using a phone to capture the spaces have been acceptable to good.
Nice! Glad to hear it’s not just for research nerds these days. I’ve done a fair bit of experimentation with photogrammetry and it does work well for the built environment.
I have an application however where I want to be able to capture an outdoor forest scene with a video camera and convert that to a video game/simulation scene which renders efficiently. It turns out that is a more challenging problem that actually no one has solved as far as I know.
There’s lots of neat research tho and it will be possible some day. This bit of research is pretty cool:
https://youtu.be/JuH79E8rdKc
Do you think its possible to decouple lighting effects from the generated 3D scene? Currently it seem to simply copy the actual lighting condition in the environment. In a way, it's a limitation. It's much better if it can provide the RGB color + other material information (SVBRDF decomposition). If that's possible, then these meshes can be used in other environments.
I think its definitely possible, though I don’t know if the algorithms exist now. You would probably need to capture the scene under different lighting conditions, and probably from different angles. But assuming there is some theoretical way to go from photos to a mathematical representation of material characteristics I’d think we’ll eventually have neural nets that can do the conversion.
