By the time readers receive this issue of the newsletter, SK Group Chairman Chey Tae-won — who personally flew to Taiwan last month for Computex and made a strong impression on local audiences by welcoming Jensen Huang at SK’s exhibition booth, and who is currently the most powerful corporate leader in South Korea — should already have arrived in New York to ring the opening bell for SK hynix’s Nasdaq ADR listing ceremony. This will be the largest ADR offering in U.S. history.

As discussed in the previous issue, the massive chain reaction triggered by the severe shortage of AI memory (HBM) continues to unfold. South Korea’s two memory giants remain at the center of global attention.

On July 7, Samsung Electronics reported astonishing second-quarter results, posting operating profit of US$58.4 billion, surpassing Nvidia’s US$53.5 billion in the first quarter and becoming the most profitable company in the world on a quarterly basis.

Profit for the quarter alone is also expected to exceed TSMC’s entire quarterly revenue (which has yet to be officially announced).

Memory, Not GPUs, Is the Real Engine of AI

All signs seem to support the “AI = Memory” theory put forward by Professor Kim Jung-ho of the Korea Advanced Institute of Science and Technology (KAIST), widely known in Korea as the “Father of HBM.”

Speaking on a local television program, Kim argued that memory read and write operations account for almost all of AI computing time, while GPUs spend most of their time waiting on the sidelines. Therefore, “memory is the real protagonist of AI.”

An analyst covering South Korea’s memory industry told me that, from a numerical perspective, he agrees with Kim’s argument.

“Over the next several years, memory will account for more wafer demand and generate more profits than GPUs.”

The origins of this dramatic shift go back more than a decade.

At the time, two camps were competing to define the next generation of memory technology.

One was today’s well-known HBM, while the other was HMC (Hybrid Memory Cube), jointly promoted by Intel and Micron with participation from Samsung.

Both architectures were designed to break through the “memory wall” and enable supercomputers. Both vertically stacked multiple DRAM dies and used through-silicon vias (TSVs) to create dense interconnects through the wafers.

To move massive amounts of data quickly, HBM’s defining characteristic was its extremely dense array of bottom-side connections. To support these connections, an additional silicon interposer layer had to be added on top of the traditional substrate in order to achieve ultra-high-density routing.

HMC was a closed specification with fewer participants in the discussion process, allowing standards to be finalized quickly. It eliminated the need for expensive silicon interposers but instead added a logic chip underneath the stacked memory for signal processing. This introduced side effects including high power consumption and heat generation, ultimately leading to the failure of the standard.

The HBM standard, developed by the international standards body JEDEC, had to be discussed and agreed upon by committee representatives from participating companies. After prolonged deliberation, the formal specification was finally released in the autumn of 2013, roughly six months later than HMC.

However, because the standard was “highly pragmatic” and offered better mass-production feasibility than HMC, it ultimately emerged as the winner.

How Did CoWoS Find New Life Through HBM?

………

(To read this exclusive story in full, visit the Tech Taiwan Substack)

Have you read?

• How ASPEED Turned an Overlooked Chip into Taiwan's Highest-Priced Stock
• Delta Chairman Cheng Ping's 700-Day Succession
• How TSMC Will Surpass 2nm and Other Insights from New IMEC CEO

Uploaded by Ian Huang