Decoding the iPhone X
Apple’s ‘Face ID’ Made in Taiwan
A main highlight of Apple’s much-hyped iPhone X is its facial recognition function (Face ID). After six months of digging, CommonWealth Magazine has confirmed that Taiwan’s Himax and TSMC are the main suppliers backing this critical technology.
Apple’s ‘Face ID’ Made in TaiwanBy Liang-rong Chen
In Tree Valley Park just outside the better known Southern Taiwan Science Park in Tainan, a high-tech factory is taking shape as cranes, cement mixer trucks and more than 100 workers work day and night.
This is Himax Technologies Inc.’s new 8-inch wafer level optics plant, at US$80 million the biggest investment ever made by the company
Himax’s core business has been display driver ICs, and at one point it was one of the world’s two biggest display driver IC producers along with Taiwan-based Novatek Microelectronics Corp.
But Himax’s latest engines of growth are tiny optical lenses smaller than grains of rice, which have positioned Himax as the newest challenger of Largan Precision Co., a leading smartphone camera lens manufacturer with the highest share price on Taiwan’s stock exchange.
These small glass lenses will be installed in the front of Apple’s newest iPhone X as a key component in the device’s highly anticipated 3D facial recognition feature – called Face ID – used to keep the phone secure.
Himax Supporting Face ID
These wafer level optics elements are the result of an extremely efficient and precise process technology developed through decades of painstaking effort and the heavy investment of resources in the semiconductor sector. Long foreseen as having the fastest production speed, the lowest costs and the highest precision, the new technology is expected to set off a manufacturing revolution in the sector.
In Himax’s headquarters, 600 meters away from the construction site, the first floor office and part of the employee cafeteria and parking lot in the basement have given way to a new wafer level optics production line that began shipping product in the third quarter.
At a recent investor conference, Himax expected that the new wafer level optics line should contribute to a 90 percent surge, or about an NT$800 million increase, in non-driver IC revenue in the third quarter from the previous quarter.
Industry insiders have no doubt about the source of this urgent spurt in demand coming at this point in time – the iPhone X.
“If anybody claims to have 3D sensing revenue this year, I think it must be from Apple, because no other smartphone has this feature just yet,” said Himax President and CEO Jordan Wu at Himax’s second quarter investor conference in August, referring to the technology behind Apple’s Face ID.
That statement indirectly confirmed that Apple is Himax’s customer, because just minutes earlier Wu stressed that 3D sensors were the main focus of shipments of wafer level optics – optical products manufactured using a semiconductor process on wafers.
Himax would not confirm this through official channels, but an executive with a vendor working with Himax on 3D sensor products confirmed that the company’s wafer level optics are being shipped to Apple. Himax has also been mum about the function of its tiny optical lenses in the Face ID module.
TSMC’s New Secret Weapon
Aside from Himax, semiconductor giant TSMC is the other key Taiwanese manufacturer contributing to Apple’s 3D sensor solution.
TSMC makes the near-infrared imaging sensor in the iPhone X’s TrueDepth camera that captures the reflection of thousands of projected dots that map the user’s face and allow the system to recognize it.
Another vendor said TSMC may be the exclusive supplier of the component or may be splitting orders with European semiconductor company STMicroelectronics. It’s a secret weapon that TSMC has been honing for a long time but that has only now appeared in the public eye.
Few know that beyond its core contract semiconductor business, TSMC also manufactures CMOS (complementary metal oxide semiconductor) image sensors for digital cameras and surveillance cameras, but it has lagged behind Japan’s Sony Corp. and South Korea’s Samsung Electronics Co. in the field.
“TSMC has spent 20 years in the business but is still No. 3,” said Liu Hsin-sheng, the head of TSMC’s sensor and display business development department, at the TSMC Technology Symposium in Taiwan in May.
But now, TSMC finally has an opportunity to climb to No. 1 in the world in the field with its near-infrared image sensors. As Liu said, “We want to overtake them.”
Why is TSMC so sure of itself? Because of a unique spectrum invisible to the naked eye – a 940 nanometer light wavelength.
What’s special about this spectrum is that it does not appear naturally in the rays emitted by the sun after they pass through the Earth’s atmosphere. Thus, when the iPhone X’s microlaser projects tens of thousands infrared dots to scan the user’s face and then reflects them back to the TSMC-made sensor, “it will not be disturbed by any other light wave,” either indoors or outdoors, Liu said at the conference.
Many in the industry are predicting that this technology will eventually be widely used in augmented reality applications, autonomous vehicles and other products related to the Internet of Things, and emerge as the main light source in the fast developing “machine vision” realm.
Liu said at the conference that in the future every smartphone will be equipped with at least three to four of these “mechanical eyes” that look at special light sources, “and based on our own information, TSMC is quite likely the leader in the field.”
Though Samsung and Sony have invested heavily in this technology, TSMC’s process has the highest quantum efficiency – defined simply as the amount of energy absorbed from a wavelength or light source. The 940nm wavelength’s quantum efficiency exceeds 35 percent, indicating that TSMC’s sensor can interpret lasers with lower rates of power consumption. That means less power used, the most important attribute for any smartphone component.
Himax designs its own near-infrared image sensors for other clients that are manufactured by TSMC, and Himax’s Jordan Wu marvels at the performance of TSMC’s sensors. “Their quantum efficiency is twice that of their competitors’ products,” Wu says.
The iPhone X will be the most expensive smartphone in history, carrying a price tag of NT$35,900 in Taiwan, and how many of the phones Apple will be able to sell remains a major question mark.
Yet Himax seems to feel extremely bullish about its prospects for the next two years. It has not only invested in a new plant that should be completed by the end of the year but also left itself enough land next door to build two more factories, and that land may be used sooner than later.
“Judging from our customers’ enthusiasm, we are planning to kick-start the phase II capital expansion beyond the phase I’s $80 million mark that we announced earlier much sooner than expected in order to fulfill the strong 3D sensing demand for the next 2-3 years,” Wu said at the second quarter investor conference.
That’s because the king of smartphone chips – Qualcomm – has joined with Himax and TSMC in developing a 3D depth sensing camera system whose features are every bit as good as that of Apple’s “Face ID.” The companies hope the new product will become standard equipment in the more than 100 million high-end Android-powered smartphones that are sold around the world every year.
“Apple’s technology is not necessarily ahead of others. In some areas, it has a lead, but in some areas we have the lead,” says Chienchung Chang, head of Qualcomm’s multimedia research and development group. A Taiwan native, Chang left Taiwan for the United States after graduating from National Tsing Hua University with a degree in electrical engineering. He joined Qualcomm after getting his Ph.D. and has been there ever since.
He also happened to attend Tainan First Senior High School at the same time as Himax’s Jordan Wu, but the two never crossed paths.
Qualcomm’s Taiwan Connection
A month before Apple’s launch of the new iPhones, Chang met with a group of reporters from Taiwan at Qualcomm headquarters in San Diego and explained the new depth-sensing technology’s functions that were nearly identical to those of Apple’s “Face ID.” Whenever a user picks up the phone and points it at their face, the 3D sensing device will identify the user’s profile in a few thousandths of a second and immediately unlock the phone.
“This is what the user interface of a mobile phone should really be like. It should unlock the moment it sees you,” he said. “You’re the master. Your pet should be able recognize you.”
Qualcomm owns the patent for one critical element of the 3D sensing technology’s architecture – the algorithm for “3D depth map generation” that maps a facial profile.
In support of the project, Himax has created “one-stop shop” hardware services. At its investor conference, Jordan Wu said the company has put together a 3D sensing “A-Team,” with some of the key components developed and manufactured by Himax itself, including the advanced optics using wafer level optics technology, the laser driver IC, the near-infrared CMOS image sensor (through cooperation with TSMC), and a key ASIC chip for 3D depth map generation.
According to sources familiar with the product’s development, the laser will be supplied by American company Lumentum, which also supplies the laser for the iPhone X, and assembly of the module will be done by Truly Opto-Electronics Ltd., a Chinese company listed in Hong Kong.
This 3D sensing total solution, which Wu called “SLiM,” or “structured light imaging module,” will likely have an average selling price of US$15 to US$20, with Himax accounting for more than half of the module’s value.
“It will be a major game changer for Himax,” Wu said. “If everything goes as planned, Himax will be looked at very, very differently next year and the year after.”
Qualcomm’s Strategy: Supporting Android
As for Qualcomm, the leader in the wireless communications field, much bigger strategic ambitions exist.
To start with, the “SLiM” Wu mentioned must be used together with Qualcomm’s next-generation Snapdragon processor. Because of that, Tom Sepenzis, senior research analyst at Northland Capital Markets, upgraded Qualcomm to “outperform” in mid-September and raised its target price to US$62.50 (it closed at US$51.88 on Oct. 1, 2017).
He suggested that both Samsung and Huawei, which have transitioned in recent years to using their own processors, will have to reluctantly switch to Qualcomm’s processor on some of their flagship phones to catch up with the iPhone X.
Another factor coloring Qualcomm’s approach is its acrimonious court battle with Apple over patent rights that has turned the two companies into fierce enemies and virtually ensures that Qualcomm will be booted from Apple’s supply chain sooner or later.
The way for Qualcomm to fight back is by helping the Android camp catch up with Apple’s newest innovation in the shortest possible time.
KGI Securities analyst Kuo Ming-chi, dubbed the best Apple analyst on the planet, wrote in a report in late August that Qualcomm’s 3D sensing hardware and software were not yet mature and that mass shipments of the “SLiM” package would not begin until 2019 at the earliest.
But Qualcomm and Himax both took issue with that projection. In a joint statement, they said mass production of the 3D depth sensing camera system was being targeted for the first quarter of 2018, only a few months after when the iPhone X was expected to go on sale.
At the Q2 investor conference held before Kuo made his projection, Wu offered similar guidance, saying that Himax was “aiming to bring our total solution to mass production as early as early 2018 to meet our customers’ aggressive launch timetables.”
Some analysts expect Chinese brands Xiaomi and Oppo to be the first smartphone brands to equip their products with the “SLiM” technology.
In this intensifying battle over new smartphone technologies, Himax and TSMC are both bound to benefit because the Apple and Android camps are both adopting their wafer level optics and near-infrared sensor manufacturing services.
Himax, in particular, is hoping that its latest wafer level optics initiative will pay off.
The technology has played a role in the company’s development of Google Glass and Microsoft’s HoloLens augmented reality headset that showed promise but have yet to get off the ground, contributing to the volatility of Himax share prices over the past four years.
With the arrival of Apple’s Face ID, Himax’s wafer level optics technology, carefully honed over the past 10 years, finally has the chance to generate explosive sales, a welcome change from the stops and starts faced in the recent past trying to create another “world first” for Taiwan.
Translated from the Chinese by Luke Sabatier