Hello, I'm Pazz.
Recently, the KOSPI seems to be really firing up with Samsung and SK Hynix.
Without Samsung and Hynix, other stocks only feel like they're at the 3500-4000 level... These two stocks are basically hard-carrying the KOSPI.
As recently emerged Agentic AI and other AI technologies are becoming increasingly memory-centric, the target stock prices of Samsung and Hynix seem to be rising without limit.
In this situation, I happened to come across Groq's LPU (Language Processing Unit) that Nvidia invested in at the end of last year, and it's quite an interesting concept.
First, everyone probably knows by now that ultra-high-speed memory, namely HBM, is essential for AI training and inference,
Since HBM is consuming memory supply, there's a serious memory shortage causing memory prices to rise endlessly, and memory companies' expansion capacity will only start increasing gradually from late this year to early next year at the earliest... Memory has become the core of AI infrastructure investment.
In this situation, Groq's LPU uses internal SRAM for inference.
SRAM is actually called the ultimate memory king and boasts performance and low power that can't be compared to DRAM. CPU cache memory is SRAM, after all. It can be made with general logic processes just fine. The only drawback is that it's much more expensive than DRAM.
Groq's LPU inserts hundreds of megabytes of SRAM internally and places the computation circuits needed for inference right next to it, showing incredible performance that is tens of times faster and consumes tens of times less power than the existing GPU+HBM combination. With SRAM and computation circuits placed right next to each other, it's inevitable. It seems identical to a kind of PIM (Processing In Memory) concept.
However, LPU's disadvantages are two-fold: small capacity per chip (hundreds of megabytes per LPU chip vs. tens of gigabytes for HBM) and high cost. It seems they're solving the capacity issue by packing hundreds of chips together, and seem to be offsetting the cost issue with absurdly fast token generation speed. If you generate tokens fast, you only need 1 unit instead of 10.
But there was a part that really caught my attention!
Namely, LPU can be made as much as needed using existing logic foundry processes. DRAM can only be made by Samsung, Hynix, Micron, and CXMT on Earth. It's completely different from logic processes.
On the other hand, there are multiple logic foundries such as TSMC, Samsung, Intel, GlobalFoundries, and SMIC.
Logic also has bottlenecks for cutting-edge processes, but not as much as memory... I hear this Groq chip was made in a 4nm process.
In other words, I thought that if memory prices become unreasonably high, it might be more economical to manufacture LPU using logic processes instead. I heard Chairman Chey Tae-won mentioned Hynix operating profit of 1000 trillion won... hehe... If it gets to that level, it might be more economical to just make chips by embedding SRAM internally using logic processes instead of buying DRAM. Since power consumption would also decrease significantly, it would be quite advantageous from a TCO perspective.
Since very detailed information about LPU hasn't been widely disclosed yet, I'm not sure how this technology will develop going forward. I think it's possible that CEO Jensen Huang might mention this technology at GTC around mid-next month.
Whether this technology will be a bomb for our country's memory industry or pass without much impact remains to be seen.
For now, I'm sharing this because I think it's good to keep the term LPU in mind.
