Having established a new market value record, NVIDIA is faced with the emergence of a young challenger from Taiwan.

In early April, MLCommons, a consortium of AI chip industry heavyweights such as Google, Microsoft, Facebook (Meta), and Amazon, announced a set of data at the MLPerf benchmark suite, akin to the Olympics of AI.

New Star #1: Tiny chipmaker Neuchips Outguns NVIDIA with Fastest GPU

Established four years ago, Taiwan’s Neuchips is a small chipmaker with just over 40 employees. The company’s first chip, RecAccel N3000, based on Taiwan Semiconductor’s (TSMC) 7nm process, achieved 1.67 times the QPS/Watt performance of NVIDIA’s fastest H100 GPU at 1060 queries per Watt, claiming top marks for efficiency and taking the chip sector by storm.

This signifies that Neuchips’ RecAccel N3000 chip can help the data centers of the biggest cloud services players like Meta and Google achieve greater query efficiency with less power.

The current generative AI wave is computationally hungry, and NVIDIA sits comfortably atop the market with a 78 percent share.

The opportunities at hand have attracted big industry names like AMD, Intel, and other startups, and even cloud services companies, to vie for a piece of NVIDIA’s business.

“No one wants to be hamstrung by NVIDIA,” asserted one unnamed systems manufacturer executive.

The company, located in the Hsinchu Science Park, was founded in 2019 by National Tsing Hua University professor Youn-Long Lin, a co-founder of Global Unichip Corp. (GUC). Its main investors include Wistron subsidiary ArFlex Corporation, Sunplus Ventures, and Powerchip Technology (PTC).

“That we’ve accomplished such a large project with such a small team proves that Taiwan is a great place to make chips,” exclaims Lin.

The startup’s first product utilizes TSMC’s most costly 7nm advanced process. Rare in Taiwan’s semiconductor industry, it is the main process for MediaTek’s smartphone chip starts.

Partnership with AMD

“NVIDIA’s process has reached 4nm. With 7nm, we’re already behind by one and one-half generations, so if we fall any further behind them, the product’s chances of succeeding are slim,” states Neuchips CTO, Dr. Joe Kao. Behind in production processes, yet surpassing NVIDIA in efficiency, points to this small company’s significant potential.

Joe Kao(Left), and Lin Youn-Long(Right). (Source: Chien-Tong Wang)

Neuchips chose to target AI chips for servers, an underserved segment of the market, with a focus on recommendation systems, which account for over 60 percent of AI inference.

“We’re taking a different route from NVIDIA,” explains Lin Youn-Long. NVIDIA is a general-purpose GPU that needs to cover a broad range, from engineering to meteorology and gaming, but is not necessarily appropriate for enterprises in specialized fields.

Neuchips is focused on specific uses, accelerating the ASIC (Application Specific Integrated Circuit)’s AI computation. In this way, by deliberately avoiding direct competition with big chipmaker names like NVIDIA, AMD, and Intel, it has a fighting chance.

New Star #2: Compiler Company Skymizer

Generative AI has accelerated business opportunities for chips, and NVIDIA’s biggest challenger is undoubtedly AMD.

On June 13, AMD unveiled the M1 300X, a new generation of chips with eight built-in processors, surpassing NVIDIA in terms of memory capacity and directly challenging NVIDIA's highest-end H100 for speed and performance. Despite this, following the announcement, AMD's stock price dropped by 3.6 percent, prompting one Tirias Research analyst to point out that it was due to the lack of major customers adopting the new chip.

"People still use NVIDIA because it has CUDA, which can handle high-level programming languages," admitted a senior executive from a systems manufacturer. Meanwhile, AMD and Intel lack a software architecture like CUDA.

Neuchips also wants to break free from CUDA’s shackles, for which the key lies in the chip's compiler developed by the Taiwanese software company Skymizer.

The day of our interview with Skymizer founder and CEO, Luba Tang, May 19, Meta had just announced its first inference chip, the Meta Training Inference Accelerator (MTIA), a custom chip that comes with its own PyTorch 2.0 interface. At the same time, OpenAI also launched the machine learning model Triton DSL, all aiming to weaken NVIDIA's dominance.

“This all took place over the last month or so,” observed Luba Tang.

Luba Tang(Left), and Jim Lai(Right). (Source: Zhu-Jun Wang)

Skymizer was established in 2013 with investment from former Intel Vice President Dr. Wen-hann Wang. The current company CEO is a well-known figure in the Taiwanese IC design industry, former Global Unichip (GUC) president Jim Lai, and CEO Luba Tang has worked at Morningstar and Marvell and was second-in-charge before MediaTek established its compiler department.

Skymizer specializes in designing compilers for ASIC manufacturers, enabling the integration of ASIC with AI deep learning and high-speed computing to enhance hardware performance. It is also the world's first open-source compiler design company, playing a role similar to NVIDIA's CUDA.

Therefore, whenever a new AI platform emerges, Skymizer must help ASIC customers "land" their chips so that the customers' ASIC chips and these AI programming languages can communicate together.

“We can become the CUDA for the non-NVIDIA camp, helping ASIC companies bridge gaps,” offers Jim Lai.

New Star #3: Alchip

Alchip specializes in the design services of large-scale AI high-performance computing chips and is one of the major users of TSMC's CoWoS production line. It competes with such chip giants as Nvidia, AMD, and Broadcom for production capacity.

Last year, Alchip's contribution to high-performance computing solutions accounted for over 80 percent of its revenue, of which advanced processes below 7nm accounted for 68 percent. This year, AI-related businesses are expected to account for 60 percent of Alchip's revenue, doubling over last year. Over the past six months, Alchip's stock price has soared by over 120 percent, earning it the label of the most attractive AI concept stock in the market.

“This is because our biggest customer grew a lot,” relates Alchip spokesman and CFO, Daniel Wang. Speculation in the market points to the company’s biggest customer being a large US-based cloud services provider.

Interestingly, NVIDIA President Jensen Huang’s biggest competitors are actually his own customers - major cloud service providers such as Google, Microsoft, Meta, and Amazon, all of whom are developing their own chips.

This group of large cloud services providers developing chips in-house has become Alchip’s ideal partners.

Alchip’s co-founders, Chairman of the Board Kinying Kwan and President and CEO Johnny Shen, both came from Silicon Valley company Altius, specializing in chip design services. Their claim to fame was securing the backend design for Sony’s PlayStation 2 chip, which accounted for 80 percent of Sony's revenue and shrunk the chip's size by 30 percent.

Since its establishment, Alchip has focused on ASICs for network communications and high-performance computing, targeting advanced processes. However, before the rise of Bitcoin mining and AI in 2018, the market’s mainstream was dominated by small-scale ASICs for mobile devices, and Alchip's revenue remained stagnant at around NT$4 billion.

At the time, a senior figure in the IC design industry reminded Johnny Shen that a chip priced above US$10 would be difficult to sell.

"But at that time, the large ASICs we were designing cost a few hundred dollars each," recalls Shen.

Alchip’s technical team has always taken great pride in designing the highest-performance chips. Including Johnny Shen, this collection of engineers from Altius did not want to compromise its know-how by giving in to the market.

In 2009, IBM joined forces with the University of Tokyo to introduce the world's fastest supercomputer using TSMC's 90nm process. It was later revealed by the media that Alchip was responsible for the design and successful production of the chip.

“After that, as long as supercomputing chips were needed, everyone in Japan, China, and even the US came to us,” recalls Johnny Shen.

Subsequently, bitcoin vendors also sought out Alchip. And top Chinese AI companies SenseTime and Cambricon also produced chips with design assistance from Alchip.

As a result, Alchip established a core R&D team in the Xuhui District of Shanghai and recruited many research and development talents of the Millennials generation. It has also expanded its presence to locations such as Wuxi, Hefei, Jinan, and Chongqing to better serve China's thriving AI companies.

However, with the outbreak of the US-China trade war, in 2021 Alchip’s shipments to marquee Chinese customers such as Soar Cloud and Goke Microelections suffered disruption as the United States imposed political restrictions. Alchip thus began shifting its focus to North American customers and soon resumed shipments.

According to Alchip's latest annual report, sales in China accounted for over 70 percent in 2021, dropping to 27 percent in 2022. In contrast, US share grew from 13 to 40 percent.

“US customers put pressure on us not to do business in China,” revealed Shen. And Alchip has already strengthened its recruitment efforts at research and development bases in Taiwan, Vietnam, and Malaysia.

Johnny Shen observes that NVIDIA's advantage, established over a decade in integrating software, chips, and networking, created a CUDA technology ecosystem of hardware-software integration along the lines of Apple, the king of consumer electronics.

“Given market trends, it will be tough to replace CUDA over the short term for AI,” admits Johnny Shen.

However, cloud services providers such as Microsoft, Facebook (Meta), and Google command vast resources and robust software development capacities, which Shen sees as “undoubtedly a rare opportunity for ASIC companies.”

---

Have you read?

• The man behind Nvidia’s lead in the AI race
• Can Taiwan Find an AI Niche? 
• AI medical records assistant Copilot 

Translated by David Toman
Uploaded by Ian Huang