Among a slew of advanced technologies promising to create tomorrow’s self-driving cars, lidar technology has gained traction. However, lidar sensors are also being used in efforts to create digital twins and metaverse use cases.

A key feature of digital twins is the ability to update models of the real world with high fidelity and frequency. Lidar technology complements stereoscopic cameras in capturing 3D data from the physical world, which can then be piped into digital twins or metaverse applications.

Core lidar technology has been around for nearly 50 years, but it has been expensive to build and difficult to integrate into new workflows until recently. All of this is beginning to change with the introduction of new approaches, less expensive implementations, and more flexible lidar data workflows.

Lidar sensors hold great promise in self-driving cars and related systems because they measure distance to objects at incredible speeds and with high precision. They function similarly to radar in that the lidar signal is reflected off objects and the light that returns is measured. Traditional lidar measures signal time of flight, but newer techniques use other properties of light to reduce costs, improve precision, or accelerate reaction time.

The Lidar Sensor Business Case

The cost of these systems has dropped from tens of thousands of dollars to tens of dollars for mass-produced sensors embedded in tablets and smartphones such as the iPhone 12 Pro. Apple has been tight-lipped about the source of its lidar, but one teardown of the first iPad with the technology revealed a laser scanner from Lumentum and a sensor from Sony combined into a lidar system.

Smaller lidar form factors will allow terrestrial lidar capture to proliferate across indoor spaces and in difficult-to-access locations, such as pedestrian pathways and corporate campuses, according to Here Technologies product management director Mark Yao. His company is leveraging significant advances in hardware, software, and artificial intelligence to develop applications that automatically extract features and 3D objects from lidar to automate digital twin data capture.

Fixed Possibility

Longer range, higher 3D resolution, and better performance in low-light situations are the main advantages of lidar over cameras. Simultaneously, lidar suffers from a lack of facial recognition capabilities, higher costs, increased data processing requirements, and a lack of awareness about the technology’s benefits.

“As a result,” ABI Research analyst Dominique Bonte told VentureBeat, “lidar is frequently used in combination with other sensors to enable high-quality and high-reliability sensor fusion, particularly in the automotive environment.” Meanwhile, ABI smart mobility and automotive analyst Maite Bezerra believes that Tesla’s ability to offer some self-driving capabilities without using lidar has squeezing the lidar market.

Updating Digital Twins on A Large Scale

However, in the world of digital twins, lidar may have a lot of room to grow. In a world where there are growing privacy concerns about facial recognition, Lidar’s lack of facial recognition capabilities could be a feature rather than a flaw. Indeed, according to Seoul Robotics CEO HanBin Lee, privacy-conscious data is critical for retail and smart city applications.

Retail stores, for example, use 3D data to track metrics such as how long the checkout process takes, how customers move through the store, which products they interact with, and for how long. In some of its showrooms, Mercedes-Benz, for example, uses Seoul Robotics’ technology to monitor how visitors interact with the vehicles on display.

Deutsche Bahn is utilizing Here lidar technology for its new Sensors4Rail railway digital twin. The data allows for the tracking of changes to objects that may have an impact on operations, such as buildings, poles, and platform edges.

Senior Vice President of Enterprise Solutions at Hitachi ABB Power Grids Bryan Friehauf believes that lidar will play an important role in precisely monitoring electric utility infrastructure. He believes that lidar is critical for gathering dynamic geographic data such as vegetation growth, erosion, and the effects of climate change.

Cologne, Germany, uses lidar to supplement a citywide digital twin. BIM building data, digital terrain model data, and high-resolution photos integrated into ESRI CityEngine are some of the other data sources. Models for simulating noise, air pollution, flooding, traffic, and energy consumption can help with planning and emergency response.

Lidar can also help scientists and engineers create more accurate and realistic digital twins of coasts and oceans for fisheries planning, assessing the impact of climate change on coasts, and engineering offshore infrastructure like windmills and oil rigs.

According to Terradepth cofounder and co-CEO Joe Wolfel, ocean exploration is another area of lidar interest. “As humans seek to understand more about the Earth’s ocean and to halt climate change and damage to 98.5% of the Earth’s biosphere, maritime data is becoming increasingly important,” he explained. Models for simulating noise, air pollution, flooding, traffic, and energy consumption can help with planning and emergency response. Lidar can also help scientists and engineers create more accurate and realistic digital twins of coasts and oceans for fisheries planning, assessing the impact of climate change on coasts, and engineering offshore infrastructure like windmills and oil rigs.

According to Terradepth cofounder and co-CEO Joe Wolfel, ocean exploration is another area of lidar interest. “As humans seek to understand more about the Earth’s ocean and to halt climate change and damage to 98.5% of the Earth’s biosphere, maritime data is becoming increasingly important,” he explained.

There Are Numerous Lidar Flavors

Lower-cost, higher-performance lidar is creating new opportunities for the technology. “Lidar technology has been following the playbook of the semiconductor industry,” Abhijit Thatte, AEye senior VP of AI and Software, told VentureBeat. “With technological advancements and mass production, lasers and their components, such as lasers, scanners, optics, receivers, computer platforms, and housings, are becoming faster, smaller, and cheaper,” Thatte continued.

The algorithms are also better at assisting lidars in determining where and when to focus in order to increase the proportion of relevant data at higher resolutions, ranges, and speeds while decreasing irrelevant data. Lidar technology has come in a variety of flavors in the market. According to Gaurav Gupta, Gartner’s VP of semiconductors and electronics, the initial lidar landscape was dominated by players such as Velodyne, which used mechanical scanning lidars. However, the focus is now on solid-state lidars.

Lidar technology consists of a single component that reliably scans a laser signal over an area and a receiver that precisely measures the response. Micro mechanically controlled mirrors (MEMS), proprietary mirrors, flash, and optical-phased arrays are all methods of beam guiding (OPA).

There are also different sensing approaches that use time of flight and frequency modulation techniques. According to Gupta, there is no clear technology winner, and he anticipates that different technologies will gain market share based on the right combination and alignment with use case and costs.

Vendors that support the various technologies include:

• Lidar that scans mechanically (Velodyne lidar, Valeo, Ocular Robotics, Ouster)
• MEMS sensors and a proprietary mirror lidar (LeddarTech, Innoviz, AEye, SosLab, Luminar)
• Lidar flash (Argo AI, Sense Photonics, Continental, Ibeo Automotive Systems)
• Phased optical array (Quanergy Systems, Baraja)
• Continuous wave with frequency modulation (Aeva Technologies, Insight lidar, Aurora, SiLC, Analog Photonics)

The Desire for Simplicity

The wide range of lidar technologies promotes competition and lowers costs. However, it can make developing lidar applications for digital twins and metaverse use cases more difficult. While most vendors support common use cases such as self-driving vehicles, digital twin applications generate unique data workflows and app development processes.

According to Yao, different processes are required for ingesting, storing, and accessing data in lidar systems. Differences can also be found in aspects such as point cloud densities, scan rates, alignment, and coordinate systems.

When teams try to combine this data with 2D imagery, the process becomes more complicated, according to AEye’s Thatte. A camera, for example, captures all pixels in a 2D image at once, whereas a lidar captures a range of pixels at slightly different times. Both object depth and motion must be considered, and things become even more difficult in low light and bad weather.

Vendors are beginning to address these issues with software development kits and platforms that aid in the unification of data from various types of lidar and camera sensors. Seoul Robotics, for example, has developed SENSR2, a 3D data integration platform that normalizes data from all sensors into a common format.

Lee believes that this could help to provide a middle ground for lidar manufacturers and companies developing lidar apps. “There are so many players in the lidar sensor game, which is fantastic because it means we can customize solutions based on budget and level of detail required,” Lee explained. “However, in order to do so effectively, we need 3D perception software that is compatible with all of these different sensors.”

Illuminating Digital Twins

Long term, lidar advancements and cost reductions promise significant gains outside of traditional automotive applications. ABI sees opportunities for technology in industries such as smart cities, industrial supply chain, and security. “As the driverless agenda continues to be postponed, many lidar suppliers, including Quanergy and Velodyne, are exploring these new use cases and markets,” Bonte said. He expects the automotive lidar market to remain subdued until robotaxi services become widely available. ABI predicts that only a small number of SAE level 4 and level 5 self-driving cars will be produced in the next 4-5 years. Analysts at the firm predict that shipments of automotive lidar will be less than 500,000 in 2025, rising to 6.5 million in 2030.

Meanwhile, ABI sees plenty of opportunities for fixed lidar applications. These include traffic monitoring and management (with increased resolution and robustness over thermal imaging); people density and flow tracking and monitoring for COVID-19 use cases; and improving industrial robot management at scale in manufacturing plants and warehouses. Lidar applications could grow alongside the numerous digital twin use cases that are emerging in a variety of industries and settings.