When Sony talked about the Vision-S at CES last year, it all looked like marketing. This year, the Gambit electric vehicle project seems to have more to do with the automotive industry.
The Japanese consumer electronics giant has multiple divisions that make the company successful in the entertainment industry. The question arises: can Sony adapt its technological know-how to the automotive market and enter the world of automotive safety?
Sony has an incredible arsenal of technologies and media resources, from high-definition imaging devices, time-of-flight cameras, spatial audio and imaging technologies to 5G devices, UI / UX for smartphones, AI and recognition technologies, PlayStation 5 and Columbia Pictures.
According to the company, the electric vehicle will “be a fusion of technology and the company's creativity” and will play an important role in “developing the future of mobility and will embrace the evolution of safety, comfort, entertainment and accessibility.” No matter how grandiose these plans may seem, it is not yet clear whether these are the expectations of automakers or such are the demand.
Take CMOS image sensors for example.
They come to mind first, as imaging is one of the most important sensing technologies in ADAS and driver monitoring systems. Moreover, in 2019-2020, ADAS image sensors have advanced significantly.
“The market has doubled in two years and in 2020 it was $ 500 million. It is expected to grow by another 19 percent over the next 5 years, ”said Pierre Cambu, chief analyst at Yole Développement. Cambu cited figures from the " State of the CMOS Sensor Industry 2020 " by Yole Développement.
Given Sony's success in the smartphone market, it might seem like the company already dominates the automotive vision market.
This is not true.
In the ADAS sensor market, Sony ranks third (11 percent), behind On Semiconductor (49 percent) and Omnivision (19 percent).
A similar state of affairs is observed in the market for image sensors for driver monitoring systems (that monitor the interior of the car). On Semiconductor is also the leader in this market.
So what's the point of what's going on? Why is Sony lagging behind in the automotive industry? Maybe the company has a trump card up its sleeve - for example, a plan to enter the electric car market after Apple?
Colin Barnden, lead analyst at Semicast Research, told the EE Times: “Building cars is so easy that Tesla does it in a tent. Making money making cars and then making money making reliable cars is difficult. ”
Sony has not announced plans to make cars.
However, the company is working on a prototype electric vehicle called the Vision-S (announced a year ago). Vision has been described as an experiment“ exploring the potential for next generation mobility ”.
A year has passed and Vision-S continues to develop - now as a project aimed at the automotive market. Sony began testing the prototype on the roads of Austria last December, according to Sony CEO Kenyichiro Yoshida at CES 2021 last week.
Sony's automotive ambitions
As some analysts have suggested, Sony could very well become an automaker.
One of the hallmarks of this is the partnership with Magna - the company's CEO announced a collaboration with Sony on the Vision-S project, which has been ongoing since 2018. It looks like the deal will not be limited to prototyping. Barnden recognizes Magna as the best supplier for Sony. Barnden also believes Magna is "the best supplier of Apple iCar components." According to Barnden, the race for supply contracts between Magna and Foxconn is in full swing.
However, we are distracted. Let's put speculation aside and deal with Sony's auto business.
For now, despite its potential in chip manufacturing, Sony has yet to establish itself as a provider of cutting edge technology for the automotive market.
However, Sony's Vision-S commercial presents the vehicle's perspectives and characteristics. The Vision-S is an elegant four-seater car rife with solutions and technology (presumably Sony owned).
The prototype includes a dashboard, digital control panel and a touchscreen infotainment system that stretches from door to door. In Car Magazine write that Vision-S is similar to a "mixture Tesla Model S, Porsche Taycan and Audi A7 (especially 21-inch wheels)".
Sony's video suggests the Vision-S is equipped with some self-driving features. According to the Sony website, the car uses 40 sensors (18 time-of-flight cameras, 18 ultrasonic radars and 4 lidars), taking into account time-of-flight cameras and cameras to monitor the driver in the cabin.
Lip reading on the way
Sony has its own vision of this area and the company claims to have a number of technologies. Their successful implementation and implementation can lead to success.
Consider interior monitoring technologies.
As Sony notes, “Sensors with time-of-flight cameras track passengers. The driver's facial expressions and gestures are also recognized to assess the level of his concentration, and warnings are triggered if necessary. "
As you can see from the many announcements at CES 2021, driver monitoring systems are already becoming a reality. Next in line are speech recognition systems that will allow a curious driver to clarify something about the operation of the car or ask a question like "what is this building in front?"
Now Sony is working on lip-reading features that, according to the company, will allow "to reliably capture and recognize the driver's speech, even in noisy environments."
In a conversation with the EE Times, Sony said that although some researchers have presented the basic concept of lip speech recognition systems, no one has perfected it. Mark Hanson, VP of Technology and Business Innovation at Sony Semiconductor Solutions of America, noted and commented on the importance of such solutions. While voice-to-speech recognition may become very popular, the reality is that "these systems won't work in windy conditions." Hanson commented, "I have two cars and both are convertibles, so I could never use speech recognition systems."
He also noted that speech recognition is difficult if children are screaming in the back seat.
We asked the question - could lip reading be the way to promote Sony salon cameras? It turned out not quite. Sony's plan to sell licenses for its lip reading software to OEMs is the company's strategy to establish itself as a "technology services and solutions" provider.
We also asked if Sony is working with any specific automotive platform to handle lip reading algorithms. Has the company considered the possibility of working with a platform powered by Qualcomm Snapdragon, which already integrates Alex's voice assistant from Amazon?
It turned out not. “There can be many implementations,” said Hanson. Given that Sony's work in this area is just beginning, the company does not yet have partnerships or collaborations to announce.
However, Hanson highlighted the wide range of integration possibilities for lip-speech recognition software. Among them, we can note its launch in the car processor, embedding in cameras or adding an additional board. All of the above methods are expected to process speech locally.
Hanson also spoke about the smart computer vision sensorwhich Sony unveiled last year. It has built-in AI processing functions. Hanson noted that "the algorithms can work in the sensor itself." "This way, a certain part of the problem can be solved right in the device ... Such a smart sensor can be programmed to recognize facial features and then transmit only lip data to the model to ensure maximum performance."
The catch is that the sensor Hanson talked about is not yet used in any car. It was developed for consumer applications and does not yet have an automotive certification.
Late entry into the automotive market
Sony's big mistake is likely to be entering the car market too late. Maybe it's because she was too obsessed with her success in the smartphone image sensor market. It wasn't until 2014 that the company officially announced plans to enter the automotive sensor market.
However, over the past few years, Sony has released many high-tech thermal imagers with cutting-edge features. Among them is the IMX224MQV , released in 2014. The company claims it is "the world's most sensitive CMOS sensor for automotive cameras." In 2015 Sony acquired Softkinetics Systems(Belgium) - the company that developed the time-of-flight technology for determining the distance. Sony then renamed Softkinetics to Sony Depthsensing Solutions Holding.
In 2017, it released the IMX390CQV sensor, which was supposed to reduce LED flicker and provide high-quality HDR shooting. Later that year, Sony introduced the IMX324 , equipped with a multilayer CMOS sensor with an effective resolution of 7.42 megapixels. Sony noted that the company's new sensor is compatible with Mobileye's EyeA4 and EyeQ5.
Phil Amsrud, senior principal analyst at IHS Markit, agreed that Sony was “late” in entering the automotive market (despite the company’s very little experience in this area).
Amsrud also noted that “On Semi has become a leading supplier of CMOS (CIS) sensors due to its industry expertise - it was previously a division of Motorola Semiconductor. That is why, at the time of purchasing Aptina, On Semi had a pedigree of work in the automotive market and CIS devices. "
What let Sony down
Sony's history and expertise in consumer electronics may have prevented the company from understanding the needs of automakers.
Amsrud noted that “On Semi and Omnivision were both focused on low resolution CIS sensors, which allowed them to support low cost applications, including devices with a resolution of less than 1 megapixel. Sony, on the other hand, came from the consumer segment, focusing on higher resolution devices and focused on developing such devices for the automotive industry, although there are very few applications suitable for them. "
Early market entry with low-resolution devices allowed On Semi and Omnivision to gain the lion's share of the market. As a result, this market was transformed into ADAS.
Yole's Cambuu agreed. "The dynamics of the automotive market is significantly different from the dynamics of the mobile one." He explained: “Since ADAS was being implemented in mid-range vehicles, the performance requirements were not very high. Reliability and price are much more important, where On Semiconductor and Omnivision win. "
Cambuu also added that “price pressure is helping players like Smartens Technologies enter the Chinese market. This factor will also slow down Sony's advancement. "
He continued the thought, “At Yole we believe Sony will only be able to gain significant market share if market demand for high-performance image sensors grows. The race for autonomy can play a role. But for now, it's clear that more investment can be made in high-performance computing systems (like the Tesla FSD) than in high-performance image sensors. The market is not driven by quality, but quantity. ”
Cambuu summed it up:“ This environment is not favorable for Sony so far. ”
What the future holds for Sony
Sony's potential as an electric vehicle manufacturer is impressive (if everything happens at the same pace). However, for the Japanese consumer electronics giant, all this remains a gamble. Looking at the Vision-S, it seems like the company is trying to bite off more than it can.
More and more observers agree that the Vision-S is not doomed to wither away as a vain Sony project.
Many vendors revealed their relationship with Sony at CES 2021 as part of the Vision-S. AImotive , an ADAS and self -driving software company, announced a partnership with Sony to “improve its self-driving software as part of its Vision-S prototype.”
In an article published TechCrunch , cites "many Vision-S partners." These companies include "Bosch, Continental, Hungarian auto start-up AIMotive, software company Elektrobit Automotive, French auto supplier Valeo, telecommunications giant Vodafone and German auto parts manufacturer ZF Group."
At the end of the article it is said that "the list of partners, which also includes map company HERE, Nvidia, Blackberry / QNX and Qualcomm, leaves no doubt that a car from Sony will one day enter the consumer market."
Sony's ambition is clear. However, there is an equally important question to which there is no answer. How is Sony's semiconductor business going to make money in the automotive market if the parent company competes with automotive OEMs? Or does Sony have a better plan - to use the Vision-S as an attractive project from which to start a dialogue with OEM and Tier-1 manufacturers? If Sony doesn't overdo it with the hype, all of this could help its semiconductor division.
- Russia's first serial control system for a dual-fuel engine with functional separation of controllers
- In a modern car, there are more lines of code than ...
- Free Online Courses in Automotive, Aerospace, Robotics and Engineering (50+)
- McKinsey: rethinking electronics software and architecture in automotive
