If someone gets their hands on one and wants to build Deno with `cargo build --release`, I'd love to get myself release-optimized executables without waiting ten minutes for my M2 to fully compile and link the project.....
Seriously. Hit me up via email.
I think what will emerge eventually are LLM architecture specific ICs.
In terms of HyperScale this leaves Ampere only serving Microsoft. ( Microsoft is also an investor of Ampere ) As both AWS and now Google has their own design. Along with the much smaller cloud vendors.
They also have their own custom core design instead of using standard ARM Neoverse Core. Historically speaking there hasn't been a single vendor that outperform ARM in terms of CPU IP Core design price / pref apart from Apple.
Would love to see how it perform against a similar 3nm Zen 5C with 128 Core / 256 Thread.
I've been looking at these a possible Mac Studio competitor for prosumer inferencing but the current models on the market are not attractive for this purpose. Expect the model being advertised here to be available to own for quite awhile.
I do sort of think this is where things are going to be headed though for inferencing - massively parallel DDR memory and a fast ARM cpu. GDDR and HBM based solutions just seem to cost too much.
Any experiences on using Ampere as a development machine?
So, is Intel just too far behind to catch up at this point?
It looks like there is a lot of progress by quite a few companies, but Intel is still floundering.
So they don't get into it until the very end, but the claim is that these things have so many cores that they're actually faster running CPU inference than GPUs while being a little bit cheaper? That'd be pretty fun if true - of all the things to put pressure on Nvidia, I would have bet on another GPU vendor before I bet that CPUs would leap back into being faster!