Microsoft wants to launch an AI chip as it looks to position itself as the Apple of AI, offering a vertically integrated solution for scale-out artificial intelligence. 

According to The Information, Microsoft has been working on the chip, codenamed Athena, since 2019 and engineering samples are already available to some Microsoft and OpenAI employees. 2019 was the year when the former invested $1 billion in the latter; so there’s a likely link between the plan for the chip and this first investment.

Other US based companies (Amazon, Google, Facebook) and non-US entities (Tencent, Alibaba) have launched their own custom-built chips for AI with the aim to get a hardware solution that performs better than what is commercially available off the shelf. These processors are essential for inference and large-language model (LLM)training and right now, Nvidia corners the market with the A100 and the H100. 

In doing so, they are mimicking Apple’s uber-successful vertically integrated, hardware-and-software stack, that’s deployed across billions of devices worldwide. Microsoft unsuccessfully attempted to follow Apple’s footsteps in the consumer market with the ill-fated Windows RT on Arm almost a decade ago but this time around, Microsoft seems to be more in control, and most importantly, not a laggard.

Saving billions 

Microsoft has, to put it mildly, embraced artificial intelligence at scale with enthusiasm. It has baked ChatGPT across all its flagship products and services (Bing, Microsoft 365, Github) and it’s only a matter of time before it gets deployed across its entire Azure stack. The biggest obstacle to the rollout is the hardware cost which, according to experts, could reach tens of billions of dollars per year. Microsoft is already collaborating with Nvidia to build a next-generation supercomputer and it is likely that the world’s largest software company may leverage Athena to get better rates or priority from Nvidia in the future.

Other than the fact that about 300 people are working on Athena at Microsoft, the Information also claims that the roadmap for the chip includes “multiple generations” and the first generation will be fabbed using TSMC’s 5nm manufacturing process. Ex-Apple CPU architect, Mike Filippo, is likely to have a hand in that project although his expertise lies more with CPU.

To some extent, Project Athena is a follow up to Project Brainwave, the FPGA-powered system that Microsoft has been using since 2017 when it started dabbling with AI. Back then, Intel Stratix 10 FPGA (Field Programmable Gate Array) cards were used to produce what now looks like the first draft of Microsoft AI strategy. Should Microsoft follow Google’s step with a TPU-like product, we can expect a lot of high-bandwidth memory (HBM), a relatively low TDP and a laser-focus on peak FLOPs per dollar.

Also check out our list of the best dedicated server and best bare metal hosting