What's the big deal with AI chips?
The future of tech could well be split into two camps.
An underground network is smuggling powerful Nvidia GPUs into China through countries like Singapore.
This story came out in the Wall Street Journal earlier this week.
The intrigue
The story begins with a Chinese student in Singapore returning home for vacation. Besides the usual assortment of clothes and shoes is something less innocuous: 6 highly coveted A100 GPUs from Nvidia.
This student eventually gets paid US$100 per GPU, a tidy sum for not much effort, though far less than its underground market value.
Hint: Each A100 GPU at normal retail starts at US$10,000 and goes up from there, depending on memory configuration. Underground-market prices are typically 2x or 3x that.
The contestation
Of course, the story is told through the lens of the US, which wants to prevent China from drawing ahead in niche areas it considers crucial. As such, it has banned the sale of powerful AI chips to China.
Certainly, many foreign governments aren't required to impose various US controls and laws; the student didn't break any Singaporean laws.
Still, understanding this contestation makes it easy to connect the dots to seemingly random decisions, such as the revoking of permission in May to sell older Qualcomm 4G chipsets to Huawei.
Huawei makes its own Ascend AI chip and could potentially be forced to divert chip-making capacity from it to make more mobile phone chipsets.
A drop in the ocean
While the report says as many as 12,500 GPUs make their way annually into China via such methods, it also acknowledges that it is a drop in the ocean.
- Meta now has 1,000,000 GPUs for AI.
- OpenAI says LLMs show no sign of plateauing.
- Elon Musk's xAI is setting up a 100,000 GPU cluster.
The next wave of GenAI training will require GPUs number in the hundreds of thousands. A few thousand makes no difference.
The future of tech
What will the future of technology look like?
If we assume that the US-China rivalry will continue and escalate, then a schism in tech is all but inevitable.
- Mobile phone platforms such as HarmonyOS Next.
- AI chips not based on Nvidia's CUDA.
- New chipmaking tech from China.
What do you think? I would love to hear your thoughts.
Middle-of-page photo: Huawei showing off its Ascend AI chip at its annual Huawei Connect conference, held in Shanghai in Sep 2019.
Nvidia launched its A100 in 2020, as part of a new Nvidia Data Centre GPU family. It took off and eventually made Nvidia a multi-trillion-dollar firm.
