This is what happens to photogrammetry reconstruction around water and other areas where there are not enough camera angles seeing underneath the bridge to determine the 3-D geometry. This is usually rectified post processed by Google or someone serious about representing these bridges nicely. Often this is a 3-D modelers time, or if you are really enterprising, you can fly underneath the same bridge with a smaller drone and have the camera pointing up so that you can get really accurate photos underneath from which to do SfM/COLMAP. I wish this was solvable by GenAI, but the whole thing of garbage in garbage out really applies here. You don’t know what the structure is of that bridge looks like underneath the roadway unless you’ve taken multiple photos of it, and no two bridges are identical. I’m sure someone could train an AI on every bridge imaginable and we could get something better?
I don’t think AI is necessary here, nor more data. You can post-process these bridges by voxelizing them and carving out the bottom. There are many heuristics to use. And with sanity checks to discard bridges the post-processor can’t handle, it could successfully clean up 80% of them.
I have built systems that turned organic meshes into voxel and sparse voxel octree representations, modified them, and produced meshes of various parameters. It is doable, sometimes you just need to dig into the academic papers for a month or so.
Probably the team just has higher priority work. Building this post-processor for bridges seems doable by one engineer over a quarter. But the bridges being represented better than they are today won’t likely sell more copies of the flight simulator. So it’s probably very low priority to fix.
Which is why KSP was updated to include a skyway near the launch site. But if you are flying under bridges, you are probably busy looking elsewhere to notice the less-than-photorealistic textures under the road deck.
I forgot to mention you could also perform terrestrial laser scanning or SLAM underneath the bridge and fuse those together with the aerial photogrammetry to get a unified model, but this is even more effort and more post process.
I have to wonder at what level you want a truly accurate representation of all bridges so that anyone could see them. Bad actors can ruin a lot of cool stuff.
> I wish this was solvable by GenAI, but the whole thing of garbage in garbage out really applies here. You don’t know what the structure is of that bridge looks like underneath
Putting a few bridges into Google images similar to the thread I quickly find photos of the underneath.
GenAI isn't a magic wand that solves cancer except a lack of data.
GenAI is garbage, although ironically this is something it might be ok at. Compared to a blob it might be able to fake it until we move onto better AI's not LLMs.
I love the “3D Gaussian Visualisation” section that illustrates the difference between photos of the mono data and the splat data. The splats are like a giant point cloud under the hood, except unlike point clouds which have uniform size, different splats have different sizes.
This all is well and good when you are just using for a pretty visualization, but it appears gaussians have the same weakness as point clouds processed with structure from motion, in that you need lots of camera angles to get quality surface reconstruction accuracy.
> This all is well and good when you are just using for a pretty visualization, but it appears gaussians have the same weakness as point clouds processed with structure from motion, in that you need lots of camera angles to get quality surface reconstruction accuracy.
The paper actually suggests the opposite. That gaussian splats actually outperform point clouds and other methods when given the same amount of data. And not just a little bit, but ridiculously so.
Their Gaussian splatting based SLAM variants with RGB-D and RGB (no depth) camera input both outperform essentially everything else and are SOTA (state-of-the-art) for the field. RGB-D obviously outperforms RGB but RGB data when used with gaussian splatting performs comparably to or beats the competition even when they are using depth data.
And not just that but their metrics outperform everything else except for systems operating on literal ground truth data but even then they perform comparably to those ground truth models within a few percent.
And importantly where most other models run at ~0.2-3fps, this model runs several orders of magnitude faster at an average 769fps. While higher fps doesn't mean much past a certain point, importantly this means you can do SLAM on much weaker hardware while still guaranteeing a WCET below the frame time.
So this actually is a massive advancement in the SOTA since gaussians let you very quickly and cheaply approximate a lot of information in a way you can efficiently compare against and refine against the current inputs from sensors.
I will believe this when I can actually measure scenes from Gaussians accurately (I have tried multiple papers worth of experiments with dismal results). No one in the reality capture industry uses splats for anything else other than visualization of water and sky heavy scenes because this is where a Gaussian splat actually renders in a nice way. I look forward to the advancements that Nerf and GS but for now there is no foundational reason why they can extrapolate any more data than COLMAP or GLOMAP when the input data is the major factor in defining scene details.
I work in the rendering and gaming industry and also run a 3D scanning company. I have similarly wished for this capability, especially the destructability part. What you speak of is still pretty far off for several reasons:
-No Collision/poor collision on NERFs and GS: to have a proper interactive world, you usually need accurate character collision so that your character or vehicle can move along the floor/ground (as opposed to falling thru it) run into walls, go through door frames, etc. NERFs suffer from the same issues as photogrammetry in that they need “structure from motion” (COLMAP or similar) to give them a mesh or 3-D output that can be meshed for collision to register off of. The mesh from reality capture is noisy, and is not simple geometry. Think millions of triangles from a laser scanner or camera for “flat” ground that a video game would use 100 triangles for.
-Scanning: there’s no scanner available that provides both good 3-D information and good photo realistic textures at a price people will want to pay. Scanning every square inch of playable space in even a modest sized house is a pain, and people will look behind the television, underneath the furniture and everywhere else that most of these scanning videos and demos never go. There are a lot of ugly angles that these videos omit where a player would go.
-Post Processing: of you scan your house or any other real space, you will have poor lighting unless you took the time to do your own custom lighting and color setup. That will all need to be corrected in post process so that you can dynamically light your environment. Lighting is one of the most next generation things that people associate with games and you will be fighting prebaked shadows throughout the entire house or area that you have scanned. You don’t get away from this with NERFs or gaussian splats, because those scenes also have prebaked lighting in them that is static.
Object Destruction and Physics: I Love the game teardown, and if you want to see what it’s like to actually bust up and destroy structures that have been physically scanned, there is a plug-in to import reality capture models directly into the game with a little bit of modding. That said, teardown is voxel based, and is one of the most advanced engines that has been built to do such a thing. I have seen nothing else capable of doing cool looking destruction of any object, scanned or 3D modeled, without a large studio effort and a ton of optimization.
I think collision detection is solvable. And the scanning process should be no harder than 3D modeling to the same quality level. Probably much easier, honestly. Modeling is labor intensive. I'm not sure why you say "there’s no scanner available that provides both good 3-D information and good photo realistic textures" because these new techniques don't use "scanners", all you need is regular cameras. The 3D information is inferred.
Lighting is the big issue, IMO. As soon as you want any kind of interactivity besides moving the camera you need dynamic lighting. The problem is you're going to have to mix the captured absolutely perfect real-world lighting with extremely approximate real-time computed lighting (which will be much worse than offline-rendered path tracing, which still wouldn't match real-world quality). It's going to look awful. At least, until someone figures out a revolutionary neural relighting system. We are pretty far from that today.
Scale is another issue. Two issues, really, rendering and storage. There's already a lot of research into scaling up rendering to large and detailed scenes, but I wouldn't say it's solved yet. And once you have rendering, storage will be the next issue. These scans will be massive and we'll need some very effective compression to be able to distribute large scenes to users.
You are correct; most of these new techniques are using a camera. In my line of work I consider a camera sensor a scanner of sorts, as we do a lot of photogrammetry and “scan” with a 45MP full frame. The inferred 3D from cameras is pretty bad when it comes to accuracy, especially from dimly lit areas or where you dip into a closet or closed space that doesn’t have a good structural tie back to the main space you are trying to recreate in 3D. Laser scanners are far preferable to tie your photo pose estimation to, and most serious reality capture for video games is done with both a camera a and $40,000+ LiDAR Scanner. Have you ever tried to scan every corner of a house with only a traditional DSLR or point and shoot camera? I have and the results are pretty bad from a 3D standpoint without a ton of post process.
The collision detection problem is related heavily to having clean 3D as mentioned above. My company is doing development on computing collision on reality capture right now in a clean way and I would be interested in any thoughts you have. We are chunking collision on the dataset at a fixed distance from the player character (can’t go too fast in a vehicle or it will outpace the collision and fall thru the floor) and have a tunable LOD that influences collision resolution.
Both my iPhone and my Apple Vision Pro both have lidar scanners, fwiw.
Frankly I’m surprised that I can’t easily make crude 3D models of spaces with a simple app presently. It seems well within the capabilities of the hardware and software.
Those LiDAR sensors on phones and VR headsets are low resolution and mainly used to improve the photos and depth information from the camera. Different objective than mapping a space, which is mainly being disrupted by improvements from the self driving car and ADAS industries
I feel like the lighting part will become "easy" once we're able to greatly simplify the geometry and correlate it across multiple "passes" through the same space at different times.
In other words, if you've got a consistent 3D geometric map of the house with textures, then you can do a pass in the morning with only daylight, midday only daylight, late afternoon only daylight, and then one at night with artificial light.
If you're dealing with textures that map onto identical geometries (and assume no objects move during the day), it seems like it ought to be relatively straightforward to train AI's to produce a flat unlit texture version, especially since you can train them on easily generated raytraced renderings. There might even be straight-up statistical methods to do it.
So I think it not the lighting itself that is the biggest problem -- it's having the clean consistent geometries in the first place.
Maybe a quick, cheap NeRF with some object recognition, 3D object generation and replacement, so at least you have a sink where there is a sink and a couch where you have a couch, even though it might look differently.
Is there a teardown mod that uses reality captured models? Or is there any video even? I have played the game once, destruction was awesome. I want to see how it looks like the way you said.
Wider implications beyond just the consumer drone market. My company and several of our clients (who are all much larger companies than us) have $100,000s - $X,000,000s of DJI enterprise drones, batteries, and payloads (Matrice 300, 350, zenmuse p1 camera, etc)
I know the bill still has to go through the senate, but this is going to be a sore subject for a lot of American companies who use DJI equipment.
No reason for the Senate to not vote it into law. There are no lobbyists telling them to do otherwise. In an election year, they desperately need to show that they can get something, anything passed.
I wonder what camera sensor was used to produce this. A DSLR on a pano/360 rig? Or something purpose built like a Weiss AG Civetta 230MP 360 scanning camera.
They also have x-rays of the individual works. Those seem to have been done in a separate process from the visible photos. Click on the camera icon next to a piece and there is then an additional selection for x-ray view.
I notice a DJI Matrice 300 (larger drone) and a DJI Mavic 2 (can’t be totally sure if it’s a pro model). DJI is well known as the top market leader in UAV space because of its relative low cost ($13K for the Matrice, under $3K for the Mavic) for high utility/reliability (something you care about if you have a payload worth $10K plus). Similar drones made in the USA or EU are 2X-4X the cost and do not come with any other features that DJI hasn’t already thought of.
I guess that being a market leader means you will have a wide range of customers using your product
The data in their nicely colored graph is wildly inaccurate. I have a long range Tesla model 3 (in Alaska) and the range is over 290 miles (it says 310 miles when full, but doesn’t quite get that, even in the summer). This is information you can trivially look up…
I know Tesla gets a lot of flak, but it’s batteries outperform most of the other cars that are listed.
Are you suggesting there is nearly zero difference in battery range in cold weather? Because that seems like something you can trivially look up as well. Heaters use power.
> One interesting thing that points out is that internal combustion engines are also about 15% less efficient at cold temperatures.
They are less efficient at cold engine temperatures, but more efficient at cold air temperatures. This means that in cold winter, once the engine warms up (which might not take place on short routes), internal combustion engines are more efficient than in summer.
That may well be, but it seems like the 15% is a high-level number that takes a lot of factors into account, such as power used by heating accessories, and the fact that cold air is denser (more air resistance).
That sounds reasonable: note that I said that on short routes the engine might not have a chance to warm up.
Your source, however, quoted this 15% figure for city driving, I think that’s worth stressing as well. I wonder if they have any data for highway driving. I’m too lazy to Google, though.
I've never seen that happen. I have seen it give optimistic numbers, which are dozens of miles short of GPS trip logs, and worse in moderate cold.
When you say "this is information you can trivially look up", not only does this article reference their own testing, but others, and a quick google shows these results aren't abnormal. This article is on the extreme end of test results, but it is not unreasonably so.
Tesla does get a lot of flak, and unsubstantiated fanboyism like this does not help.
To clarify, yeah, the yellow "70f" bars are just completely wrong, at least from the Teslas I'm familiar with.
It claims a Long Range Model 3 like we have (310mi range) gets 215 miles, a 100kWh Model S about the same, and a 75kWh Model X barely over 150.
They don't list the methodology for the yellow bar, but since it's not experimental (they're comparing estimated and actual winter range with the dotted blue vs solid blue bars), presumably the yellow should be rated range, not tested range.
EDIT: Oh, now I see. If you scroll down to the car, they are, in fact, testing real world range at all temps. Claiming a long range model 3 gets 60% of its rated range at 70f is a bit mind-boggling, though. Ours has over 70,000 miles on it and while a full charge is now rated at more like 290mi, you'll certainly get over 200, even at 80mph.
Sounds like a lot of variance in experience, even in this thread. But if your car isn't breaking down, sounds like you should keep using it! But also be aware of inherent differences in battery performance in cold weather when recommending EVs to other Alaskans
We have a tiny electric fiat 500 for small distance trips. 21 kWh battery. My wife routinely gets 7 km/kWh out of it, I struggle to reach above 5.5 km/kWh.
Driving style really matters. Especially with such a small battery ;)
