How Apple Made Affordable iPhone Lidars Without Moving Parts
At the iPhone 12 presentation on Tuesday, Apple unveiled the new lidar capabilities. According to Apple, lidar will improve the iPhone's camera by making focusing faster, especially in low light conditions. Perhaps in this way a new generation of sophisticated AR applications will appear.
On Tuesday, the presentation covered how lidar works in the iPhone, although this is not the first device with Apple's lidar. The company first unveiled a device using this technology in March with an updated iPad. While no one has had time to disassemble the iPhone 12 yet, we can learn a lot from the recent teardown of the latest iPad.
The principle of lidar is that it sends out laser light and measures how long it takes to return. Since light travels at a constant speed, the round trip time can be converted into an accurate estimate of distance. Repeat this process on a 2D grid, and the result is a 3D “point cloud” showing the location of objects around a room, street, or other location.
A June analysis by System Plus Consulting showed that the iPad's lidar sends light using a vertical emitting laser array (VCSEL) manufactured by Lumentum. It then captures the flashback using a single photon avalanche diode (SPAD) array supplied by Sony. I'll explain what it is in the next section.
I found Apple's presentation particularly interesting because I was working on a text about companies that are using these technologies (VCSEL and SPAD) to create a much more powerful lidar for the automotive market. Vertical-emitting lasers and single-photon avalanche diodes are interesting in that they can be mass-produced using conventional semiconductor manufacturing techniques. Thus, the benefit arises from the huge savings in high volume production. As the adoption of vertical laser sensors grows, their quality will increase (and the price will decline).
The two companies working on top-end vertical laser lidar (Ouster and Ibeo) are already getting more support than most players in the tight lidar market. Apple's decision to adopt this technology (and the possibility that other smartphone makers follow Apple's lead) will provide them with a tailwind in the coming years.
Vertical lasers allowed Apple to create very simple lidars
Velodyne pioneered the lidar market with the 64-laser sensor
The first 3D lidar was introduced by Velodyne more than ten years ago. The rotating device cost about $ 75,000 and was significantly larger than a smartphone. Apple had to make lidars cheaper and smaller so they could fit into iPhones, and vertical-emitting lasers allowed the company to do just that.
What is a vertical laser? If you create a laser using traditional semiconductor manufacturing methods, you have two main technologies. You can make a laser emitting light from the side of the substrate (these are called edge-emitting lasers) or from the top (vertical cavity surface-emitting lasers - VCSELs).
Edge-emitting lasers have traditionally been more powerful. Vertical lasers have been used for decades in devices ranging from optical mice to network transmitters. It was believed that they were not suitable for high-tech solutions where large beams of light were required, but the technology developed and vertical emitting lasers became more powerful.
A small notch in the plate is usually required to expose an end-beam laser emitter. This increases the cost and complexity of the manufacturing process and limits the number of lasers that can be placed on a single plate. In turn, vertical emitting lasers emit light perpendicular to the plate, so they do not need to be cut or packaged individually. Thus, one chip can contain hundreds (or even thousands) of vertical emitting lasers. In general, when produced on a large scale, an IC with thousands of vertical-emitting lasers can cost no more than a few dollars.
It's the same with single-photon avalanche diodes. As the name suggests, they are sensitive enough to detect a single photon. High sensitivity means they suffer from noise. In order to use such diodes in devices such as lidars, complex post-processing is required. The big advantage of single-photon avalanche diodes is that, like vertical-emitting lasers, they can be manufactured using conventional technologies, and thousands of such diodes can be placed on a single chip.
The combination of vertical lasers and single-photon avalanche diodes can significantly simplify the lidar design. Velodyne's original 3D lidar contained 64 individually packed lasers on a rotating setup. Each laser had a corresponding detector. The complexity of this design and the need to precisely fit each laser was one of the reasons Velodyne was so expensive.
Recently, some companies have experimented with the use of small mirrors to "direct" the laser beam according to a given scanning pattern. In this design, just one laser is enough, but it still needs a moving part.
In turn, Apple, Ouster, and Ibeo make lidars without moving parts. Chip lidars with hundreds / thousands of vertical-emitting lasers can use separate lasers for each point in the sensor's field of view. And because all of these lasers are pre-packaged on a single chip, these devices are much easier to assemble than Velodyne's lidars.
The latest iPhones used another 3D sensor called TrueDepth Camera to enable FaceID. The module also reportedly used a vertical-emitting array of lasers from Lumentum. The principle of operation of TrueDepth is to project 30,000 points onto the user's face to form a three-dimensional model and compare the saved model with the received one (taking into account its deformations).
The lidar in the iPad projects far fewer dots than the TrueDepth camera. Video from iFixIt, captured with an IR camera, showed the lidar projecting a grid of several hundred pixels. At the same time, if the TrueDepth sensor recognized the depth based on the shape of the light falling on the person's face, then the lidar in the iPad measures the distance directly, timing the time it takes for the light to fly to the object and back. This approach is likely to provide a higher accuracy of depth measurement, and also increases the range of the sensor.
Higher power lidars also use vertical emitting lasers (VCSELs) and single-photon avalanche diodes (SPADs)
Ouster OS-1 and OS-2 Lidars The
performance of Apple's lidars is much lower than that of top-end sensors made by lidar companies. Velodyne, the company that released the first 3D lidar, claims its sensor can operate at 200 meters, while Apple's sensor works at 5 meters.
Many lidars using vertical-emitting lasers are more powerful than the sensors used in Apple devices. For example, Ouster's most powerful lidar based on VCSEL lasers boasts a range of about 100 meters at 10 percent reflectivity.
All current sensors from Ouster are similar to rotating devices from Velodyne. They use chips that have 16 to 128 vertical-emitting lasers - these chips are installed in columns on a rotating base. The simplicity of this one-piece design has allowed Ouster to reduce the price of its devices and become one of Velodyne's biggest competitors. However, these sensors still cost thousands of dollars - too expensive for use in cars, let alone smartphones.
Last week, Ouster announced plans to release a new solid-state lidar with no moving parts. Rather than lining up 16 to 128 lasers, Ouster's new device will use 20,000 vertical-emitting lasers arranged in a two-dimensional grid.
Ibeo is pursuing a similar strategy and could outperform Ouster. Ibeo has developed the very first lidar ever delivered to the mass market - the sensor for the Audi A8. It was an absolutely primitive device with a resolution of only 4 vertical lines. The company is currently developing a new device called IbeoNext. This model will have a 128 x 80 pixel laser grid - slightly smaller than the projected sensor from Ouster, but significantly larger than the latest devices from Ibeo. The company claims that its new sensor will have a range of 150 meters and a 10 percent reflectivity.
The last player worth mentioning is Sense Photonics, the company we talked about back in January... Like the other companies we've talked about, Sense uses vertical-emitting lasers and single-photon avalanche diodes in its lidars. At the same time, when working with Sense lasers, they use a technology called microtransfer printing. With its help, lasers can consume more energy, not overheat and remain safe for human eyes. So far, Sense's devices have not been very long-range, but CEO Shauna McIntyre told Ars that the company is looking to develop a sensor that works with a range of 200 meters - the Sense device will announce in early 2021.
Lidars will soon burst into the automotive market
Ibeo's lidar
Ibeo, Sense and Ouster are releasing new low-cost models as they expect a surge in demand from the auto industry. Lidars can significantly improve ADAS systems.
For example, many believe Tesla has some of the most advanced ADAS systems in the industry. At the same time, the company has a constant problem - its cars crash into stationary objects, sometimes with fatal consequences. Lidars are better at detecting stationary objects than cameras and radars, which means that the introduction of lidars can prevent many accidents, making ADAS systems more useful for drivers.
Until now, lidars were thought to be too expensive for the automotive market, but this is changing. Several companies are promising to release lidars for less than $ 1,000 over the next few years.
Ouster plans to prepare its ES2 sensor for mass production for the automotive industry in 2024. The company says the device will
start at $ 600 initially and will drop to $ 100 in the future. Ibeo has not announced a pricing for the IbeoNext, but the company says it has already struck a deal with Great Wall Motors (a major car manufacturer in China) to begin series production in 2022. year.
Companies that do not use vertical light lasers have also flocked to this market. One of the most prominent companies in this pool is Luminar, which announced a partnership with Volvo in May. Volvo plans to launch cars with Luminar lidar in 2022.
All of these designs have their own strengths and weaknesses (and different). So far, Luminar boasts a significant range - as much as 250 meters. Perhaps this is because Luminar uses lasers with a wavelength of 1550 nm, which is far beyond the visible light range. The liquid in the human eye is impervious to such light, which means Luminar can use powerful lasers that will not harm human eyes. Also, Luminar lidars have a wider field of view than Ouster devices.
The biggest question for Luminar is whether they will be able to meet the stated price of $ 1,000. When I interviewed Austin Russell, CEO of Luminar two years ago, he said that Luminar would need to “get the price down to a few thousand” in order to reach the mass market. I then assumed the Luminar lidar was worth more than "a few thousand." Now the company claims that the price of their lidars will drop below $ 1,000.
Ouster and Ibeo have no problem making their devices cheap. Companies are likely to have a problem reaching 200 meters, which is believed to be necessary to operate at highway speeds.
“Vertical-emitting lasers are inferior in brightness to those used in conventional lidars,” Ouster CEO Angus Pakala told me. "If you create a physics model, connect a single-photon avalanche diode and a vertical-emitting laser, the result is poor performance." However, Pakala said Ouster has developed a number of “fundamental solutions at different levels” that could make the combination work. Pakala said these solutions include "exceptional" suppression of out-of-range light and "placing signal processing devices near diodes" to help distinguish returning laser light from noise.
Thus, in the coming years, Ouster, Ibeo and Sense will face a big challenge: to develop the performance of the combination of vertical-emitting lasers and single-photon avalanche diodes to such an extent that their devices can operate with a range of 200 meters. If they succeed in solving this problem, then the low cost and simplicity of the chips will give these companies a decisive advantage. If they fail, they can go down to a lower level of this market.
- Russia's first serial control system for a dual-fuel engine with functional separation of controllers
- In a modern car, there are more lines of code than ...
- Free Online Courses in Automotive, Aerospace, Robotics and Engineering (50+)
- McKinsey: rethinking software and electronics architecture in automotive
About ITELMA
List of useful publications on Habré