Introduction to Establishing Vehicle Speed from Surveillance Video
Lou Peck presents a brief introduction to establishing the speed of a vehicle from surveillance video using a laser scanned exemplar point cloud, PhotoModeler, and CloudCompare. A detailed tutorial can be found here.
A rough transcript of the video can be found below.
(00:09):
Hey, everyone, this is Lou from Lightpoint Scientific and I'm going to just show you real quickly how we get vehicle speeds from surveillance video. There's going to be a much more in-depth tutorial coming from Ben Molnar that will detail the real specifics of the process in a much longer tutorial. But I just want to get everybody acquainted with the process. And those that know photogrammetry well already will probably be able to make use of the techniques that I'm going to show you right now. Those that aren't super familiar with photogrammetry are going to need to either refer to Ben's entire tutorial. Or about annually every year or maybe twice a year or so, I offer an advanced photogrammetry course for collision reconstruction and we go over the process in there as well.
(00:52):
So, this is a video of a vehicle. There's going to be a blue sedan coming through the screen. And the goal here is to determine the speed of the vehicle as it goes through the screen. So, here the vehicle speed was relatively consistent. This is just a test run that we did, that we had a VBOX Sport on the sedan as it was going through. So, we know it's actual speed, and we're just going to do a photogrammetry analysis to figure out what the speed is using PhotoModeler and CloudCompare as well. So, the first step is to export the frames that you're interested in. So, depending on how in-depth the analysis needs to be, how much money is available for your analysis, you could export every frame, every fifth frame, every 10th frame, depending on how quickly the vehicle's speed is changing.
(01:39):
So, in a situation like this where the vehicle appears to be going through pretty consistently, you might just export five frames and analyze those and look at the speed trays throughout. For the purposes of this tutorial, we're just going to export two and take a look at it. So, one of the key things, of course, when you're doing a video analysis is not just the photogrammetry, it is understanding when those frames were written, and especially with respect to each other and how healthy they are. So, that's a different thing that I'm not going to talk about right now. I'm just going to talk about the photogrammetry side of it. So, once you've exported your frames, you're going to bring those into PhotoModeler and you're also going to import the point cloud. So, this method relies on you going out to the collision site, and firing off some laser scans and then bringing that scan data into PhotoModeler, and then selecting control points.
(02:35):
So, that's what all these little gray spots are here. These are control points from the site scan data. And we're basically telling PhotoModeler what the 3D coordinates of this dash line are and of this little feature in the building, what those 3D coordinates are. And when you select about five of those or so, it's going to calculate the camera's location and the camera's focal length. And then as you select more points, once you start getting over 10 and if they're spread out pretty well across the frame, then you can also calculate what the distortion of the camera is, which is really useful for improving accuracy. At that point, once you have those control points selected, you can fade the point cloud in and out, which is one of the real benefits of having PhotoModeler Premium. You can fade the point cloud in and out and analyze that fit, and it just gives you extra confidence so that you can feel good knowing that you have a good solution and the mathematics under the hood are going to be sound.
(03:37):
On top of that, I really like to be able to do the point cloud fade in and fade out, because it makes a great exhibit to show the jury so they understand what the photogrammetry analysis is without you having to talk real specifics about computational photogrammetry analysis and how it's performed. It's always something that I try to avoid when I'm on the stand. I'd rather give them some sort of visual. So, here we have the point cloud from the site inspection over on the imports and coordinates pane. And then I'm just going to go onto the visibility and turn on point clouds right here. That's on now. And you can see right away it brings it in. The point cloud is now overlaying on top of that frame. Here we're looking at frame 421. I recommend naming those frames in a manner that keeps you organized.
(04:32):
So, now I'm going to go from clear and I'm just going to slowly make that opaque, just so you can start to analyze the fit and ensure that everything's aligning properly, that the distortion is calculated for or accounted for, and that the camera location and everything seems to be spot on. So, like I said, that helps you figure that out. But also, you can make an animation pretty easily in some sort of video analysis program or just export each one of these step-by-step and compile them to just show the jury like, hey, here's the frame that we have from the surveillance video. The car's visible in there, we went out, we did a site inspection with this fancy scanner. We are now going to fade in the measurements that we got when we're at that site inspection, just to know that we're in the right location, everything's lined up well and now we can essentially start tracing measurements on top of that.
(05:29):
So, here I'm going to turn off that background point cloud and start focusing on the car itself. So, we're going to do a very similar process, reverse projection, but we're going to be using a 3D point cloud to figure out the location of this blue sedan. So, to do that, we go into CloudCompare and we bring in the site point cloud, which you can see here, of course. And then we also bring in an exemplar point cloud of the vehicle of interest. So, in this situation, the camera was here in this portion of the parking lot. And what we do at this point is just grab the exemplar point cloud and we take a first crack as far as where we think it probably was in that frame.
(06:13):
You want to make sure that it's aligned on the roadway. Here I'm just setting my rotation point to make this a little bit easier. You want to make sure that it's on the roadway in the lane position that you think is appropriate, and that you could just grab this and move it around to wherever you think that is and then export it from here. We're using a really highly sub sampled point cloud for this portion of the analysis, just to expedite the iterations and make it really easy to move in between programs, because this can be a little bit of a tedious process. So, once you have a good understanding of where that point cloud should be, you've taken your first crack at it, you'll export from CloudCompare.
(06:54):
And to do that, you just click on the cloud, hit file save, and then save it as a PTS. And PhotoModeler's really good about bringing in all different types of file formats, including point clouds and PTSs. So, now that you bring that in and you turn on that point cloud and see how it aligns with the pixels that are representing the car's location for that frame. If you're off a little bit, then we go back into CloudCompare, iterate, move it around, and then bring it back into PhotoModeler until things align really well. So, Ben organized this project and he did a really good job. You could see he's got the point cloud in PTS format for every frame that he analyzed for that full length tutorial. And here he's got 421 and we can see this photo is called 421.
(07:39):
So, this is frame 421, and this is the point cloud that should align with that. So, if we turn that point cloud on, and then let's just mess with the opacity. And you can see as we turn that point cloud opacity up, we can see that it aligns really well with the pixels for that vehicle, for that frame. So, now we know where that car was during that frame. And if we have a subsequent frame that's say half a second later or so, and we can perform that same analysis again, then we'll know where the vehicle was a half a second later. And then you can, of course, calculate the speed based on those two positions. A lot of the times we'll also export those positions and then bring them into Virtual CRASH or some other simulation suite, and then make sure that at each time that we know that the frame is written, at those times, the vehicle in the simulation is right on top of those point clouds. And that helps you solidify what the dynamics are.
(08:41):
So, one more trick that really helps this process, I learned this from PhotoModeler a couple of years back. If you go to properties of selected photos and then go to image file name and click there, and then you can just click replace photo and retain orientation. So, when you do that, then you can select... So, we had 481 was another frame, we'll just click 481. Now everything stays the same, all the mathematics stay the same, but now the blue sedan's in a different location. So, this is 481, and let's just take a peek at Ben's fit on 481. Turn on 481 point cloud and turn up the opacity, and you can see that it fits really well from there. So, that saves a lot of time and headache when you're analyzing a surveillance video from a stationary camera.
(09:29):
You could just replace the images and retain all of the hard work you did when you're selecting control points and accounting for distortion and whatnot. So, a really cool process, makes for great exhibits. When you're presenting this to a jury to see their response to it, it's a lot of fun, or a client for that matter. They understand where your time went and they understand what the process is, even if they don't understand the underlying mathematics, which few people do. But if you can just show them what you're doing with a point cloud, how you're getting things to align with the pixels from the surveillance video, I think it goes a long way towards gaining their trust and for them to think that you're a credible analyst.
(10:18):
And it just gives them something fun to look at too, at trial, which sometimes is just half the battle, keeping their attention. So, anyway, that's the process. Like I said, Ben will have, I think it's going to be about an hour, an hour and a half long tutorial detailing all the specifics of it and how he goes about it. And hopefully, that gets you everything you need. But if you have any questions at all, feel free to shoot us an email or give us a call. Thanks.
