Security in 3D: LiDAR and Other Sensors Allow the Creation of Rich, Video Game-Like Digital Maps

Digital twin, LiDAR, sensor fusion, mixed reality and gamification – these are all terms that are more familiar to the IT, autonomous vehicle and gaming industries. But a 3D surveillance control system (3DSCS) brings these cutting-edge technologies to the usually conservative security industry by delivering real-time 3D data and superior intruder detection in an intuitive way.

A 3DSCS works with rich 3D maps (models) that users are able to move around in just like in a video game. Add to this the elimination of the sometimes difficult search for the exact location of alarm events on a 2D map and the true power of the technology becomes compelling.

Accurate 3D models of large areas can quickly and easily be created for the 3DSCS using aerial photos captured by a drone. Inexpensive commercial drones can perform automatic flight missions over an area of interest using Google Maps for reference. Through photogrammetry techniques, a 3D model is then built from the photos.

Flight missions usually take 15 minutes to one hour, and the automated generation of a 3D model of a site takes from two to three hours. The result is a full-color 3D map of the area that is a digital twin of the actual environment to be protected. 3D models of various objects, such as trees, bushes, streetlamps, fences, etc., can be added to further enhance the map. If the 3DSCS user is using a 3D engine, as in computer gaming, it is possible to further enrich the viewing experience with shadows and other visualization effects.

The key to the entire process is that the 3D models are true to scale, so whatever is measured within the map corresponds to reality. This highlights the essence of why a 3DSCS is based on 3D maps. It is not simply because they are visually attractive, but because the map serves as an internal universal coordinating system. Each point within the map has its own spatially correct X, Y, Z position relative to the center of the grid upon which the entire map rests. This allows for coordinated interaction between all components of a security system, i.e., detectors, cameras, 3D zones and even intruders. It also eliminates the complex manual setting up of presets for rotating cameras to enable coordinated interaction with a detector. The 3DSCS knows exactly where an intruder is located based on LiDAR detection and knows which pan-tilt-zoom (PTZ) cameras should be directed toward the alarm event.

A 3DSCS also provides a powerful tool for security system design through the use of advanced virtual planning in the digital twin environment. It is very easy to insert a virtual rotating camera into the 3D map and then perform tests for coverage.

Security camera coverage can be presented as a green light source so that camera blind spots can be understood by projecting both green light and shadows. This gives the user instant information on how well a given area is covered by cameras and whether they may need to add more cameras or change locations. LiDAR detectors can also be added to the 3D map and validated in the same way.

Since the entire system is three-dimensional, the detection method is also volumetric. In order to better understand exactly where an alarm event is occurring in 3D space, detectors are needed that provide this information. This is why a 3DSCS uses LiDAR detection technology, which is also found in self-driving cars.

LiDAR detectors send beams of invisible laser light into space. The reflected laser pulse is returned to the detector, and the time-of-flight of each pulse is measured to calculate the range of the object from the detector, up to a distance of 250 meters with an accuracy of 2 centimeters. These LiDAR detectors usually rotate 360 degrees and fire their eye-safe lasers with a horizontal resolution of 0.1 degree. This means 3,600 beams per rotation, up to 20 times per second. They, therefore, make millions of measurements per second, creating a “cloud” of points that the 3DSCS uses to detect static and moving objects.

There are several types of LiDARs on the market. The most common are spinning “puck” and/or solid-state LiDARs. However, more sophisticated sensor fusion models have, in recent years, been introduced which combine LiDAR with dual panoramic RGB cameras and thermal imaging. A mature 3DSCS should support all commonly available LiDARs.

So which LiDAR to choose? How many will be needed? How high and where should they be placed? These are standard design questions that can be easily answered by using the virtual planning tool described above. Just as with a security camera, a user can drag and drop the selected LiDAR and insert it into the 3D scene. Once the technical specifications are understood (range, field of view, etc.) a 3DSCS can immediately display a snapshot showing the coverage of the laser beams. This makes it very easy to determine if the detector “sees” the area to be protected. Adjustments are easily made in the virtual environment prior to actual installation.

If the designer is satisfied with the virtual placement of both the cameras and detectors, the entire design can be validated by inserting a virtual intruder. This part is not unlike a computer game, but it is a very powerful tool. On the body of this virtual intruder, laser beam “hits” are displayed and it is possible to see whether or not there will be sufficient detection in a given area, even if the intruder is attempting to evade detection by crawling around in critical places, such as in front of building entrances. The location and type of LiDARs can be adjusted as needed to achieve the desired quality of detection.

Another advantage of a 3DSCS is the design of the detection zones themselves. By clicking on the map, 3D zones of different shapes, sizes and alarm parameters can be created in seconds. An alarm event will only occur if a moving intruder enters this zone. And how does the system detect what is a moving object? The LiDAR creates point clouds and monitors the entire scene precisely – both static points and moving points. A 3DSCS can group these moving points into objects (known as “bounding boxes”) using a clustering method. The user can then define the minimum and maximum size of objects of interest. For example, objects smaller than 70 centimeters, such as cats and rabbits, or objects larger than 6 meters, such as a passing train, can be ignored. In addition to detection zones, a 3DSCS can be equipped with other types of zones, including pre-alarm zones, loitering detection zones, delayed zones, entry zones, and zones that monitor the safe distance between objects of different classes.

The system is designed so that it manages everything itself. The operator is more of a spectator and supervisor. The moment an intruder enters a detection zone, an alarm is triggered. The operator can observe all intruders on a monitor in real time and can see their positions, their sizes, their movement speed, and the trajectories of their movements. If the overall security system is equipped with PTZ cameras, these automatically slew to view the intruders.

The system stores all breach information, including camera footage, in a forensic archive. A 3DSCS can also control downstream devices, such as sirens, lights, barriers and gates. Video management systems (VMS), physical security information management (PSIM) systems, and other supporting software can instantly be made aware of the alarm. Importanly, a 3DSCS can be added to an existing security system, so there is no need to rip and replace.

The system can be set up so that, when an outdoor zone is breached, an alarm occurs and the threat level is raised. In this case, the operator/security guard immediately handles the situation according to normal procedures. If three or more intruders enter the zone, the system can automatically raise the threat level even higher and automatic actions can be intiated, such as closing entrances, locking sites containing operators, calling the police, and switching on lights and sirens.

The highest threat level can be automatically triggered by a larger detected object in a certain zone or an object moving at a higher speed. This may be a vehicle that has entered the guarded area. If the automated actions are set up appropriately, the premises can be protected very effectively.

What about false alarms? Outdoor detection systems can be susceptible to being triggered by animals and swaying vegetation. A 3DSCS can combat this in several ways, the simplest being to put the tree or shrub in question in an exclusion zone so that any movement in this area is ignored.

Alarm handling is performed by the operator by clicking on the dialog box. The system may require a comment on the alarm event, or it can be set to automatically clear alarms after a certain period of time. This is particularly advantageous for unmanned premises. The alarm is recorded but does not hang in the system for hours or days. The system can also “age” objects. This is a practical aid in cases in which a new object, such as a car being parked, enters the guarded area. This will, of course, trigger an alarm. But the system can recognize that the object has stopped moving and, after a user-defined time, it can place it in the scene and not react to it anymore – that is, until it moves again. These are typical functionalities for the residential segment, where there is no full-time operator.

Further, a 3DSCS is ideal for residential security because it can intelligently create perimeter and volume detections. For example, it can secure the area around swimming pools, including a virtual path from the house to the pool, and seal off the rest of the location. Or, conversely, it can deactivate everything during the day and leave only the pool protected to detect children who might fall in.

LiDAR detection is resistant to rain and snow, it does not mind high or low temperatures, and the time of day or night does not affect detection. Unlike conventional 2D security cameras, it can perform detections in total darkness.

The only thing that limits the range of LiDAR detection is very dense fog. But even this does not disable the whole system, which operates with infrared barriers. A LiDAR system is both the source of the light and the receiver of its reflection. In dense fog, this visibility may be reduced, but the system informs the operator about it and the detections continue, albeit with a shorter range. The system also actively monitors for deliberate obscuration of the LiDAR, with the threshold at which it is unacceptable set by the user. Once this level is reached, a sabotage alarm is issued.

Rotating cameras can be connected to the system and their rotation can be intelligently controlled. Communication with the cameras is via ONVIF protocol, so there is no need to alter an existing VMS. A 3DSCS also includes its own media server, so camera recordings can be stored directly in the system. Both LiDAR points and camera recordings belonging to a given alarm event can be played back in the archive. The standard is that the camera stream goes to both the VMS and the 3DSCS.

3D surveillance control systems are currently being deployed worldwide to protect residences as well as critical infrastructure. In coming years, they will play an increasingly larger and more important role in the evolution of physical security.

Martin Vojtek (martin.vojtek@tacticaware.com) is the CEO of TACTICAWARE, part of Hexagon Geosystems (3dsurveillance.hexagon.com).