d-Matrix vs Nvidia: Rival or Nvidia's Newest Partner?

This morning I typed three words into Gemini: Dmatrix vs nvidia. No question mark, no context. I wanted to see if a startup I kept hearing about could actually take on the company that sells almost every AI chip on the planet.

Gemini gave me a confident answer with a comparison table, a list of strengths on each side and a closing line I liked. It said d-Matrix is not a "GPU killer" but an inference accelerator that can sit alongside Nvidia. It even described how that would work: Nvidia handles one half of the job, d-Matrix handles the other, inside the same data centre.

That sounded like a prediction to me. So I checked. It is not a prediction. It already happened, six weeks ago. And while I was checking, I found that the most impressive number in Gemini's table was twice as big as the one d-Matrix itself publishes.

Why your chatbot types one word at a time

Before the comparison makes sense, you need to know what d-Matrix is trying to fix. I did not really understand it until I worked through it myself.

When you send a message to ChatGPT or Gemini, the model does two different jobs. First it reads your whole prompt in one go. That part is called prefill, and it is heavy on maths. GPUs are brilliant at it. Then it writes the answer, one token at a time. A token is roughly a word or part of a word. That second part is called decode, and it behaves completely differently.

For every single token it writes, the chip has to pull the model's weights out of memory again. A big model has billions of them. The multiplication itself is fast. The fetching is slow. So the chip spends most of its time waiting for data, not doing maths. Engineers call this the memory wall.

That is the whole idea behind what d-Matrix calls Digital In-Memory Computing. Instead of shipping the weights from memory to a separate processor, it does the multiplication right inside very fast SRAM, the same kind of memory used for a chip's cache. The data barely moves. Gemini explained this part well, and I think it is the right way to understand the company.

What each side actually built

d-Matrix was founded in 2019 and is based in Santa Clara. Its current product is a card called Corsair. It slots into a normal server through PCIe, the same kind of slot a graphics card uses, and it is air-cooled. Each card has eight small chiplets made on TSMC's 6 nm process.

Nvidia needs no introduction. The H100 was the chip of the ChatGPT boom. The B200 is the current Blackwell workhorse. They use HBM, a stacked memory that sits right next to the GPU and is among the hardest parts of the whole AI supply chain to get hold of.

Here is Gemini's comparison, rebuilt with figures I could trace back to the companies or to technical write-ups. Where I had to correct it, I marked the fix.

300 TB/s, or 150?

Gemini said Corsair has around 300 TB/s of bandwidth per card. When d-Matrix launched Corsair in November 2024, its own announcement said 150 TB/s. That is still a huge number. It is roughly 19 times what a B200 offers and about 45 times an H100. But it is half of what Gemini told me, and in a post about memory speed, being off by two times on the headline memory speed matters.

Now flip the chart around. Bandwidth is how fast the corridor is. Capacity is how big the storeroom is. And here Nvidia is miles ahead.

Gemini described Corsair as having "massive on-chip SRAM." Two gigabytes is massive for SRAM. It is tiny for a modern language model. A 70 billion parameter model needs tens of gigabytes even when it is squeezed down, so it has to be split across many cards or spill into the slower memory. That is the real trade-off, and I think Gemini's table made it sound like d-Matrix had simply beaten the memory wall. It has not. It has moved the wall somewhere else.

Training, software and size

On this part I agree with Gemini completely. d-Matrix cannot train a frontier model and does not try to. Nvidia GPUs handle pre-training on clusters of tens of thousands of chips, plus vision, simulation and almost everything else.

The bigger moat is software. CUDA has had close to twenty years to settle into every AI lab, university and startup. When a new model comes out, it runs on Nvidia first. Anyone choosing a different chip has to ask whether their engineers will spend weeks getting things to work. For most companies, that cost alone kills the switch.

d-Matrix's pitch is narrower. It says that for the specific job of talking back quickly, its hardware is far cheaper to run. At launch it claimed up to 10 times faster interactive speed, 3 times better performance per dollar and 3 times better energy efficiency than GPUs, and about 2 milliseconds per token on Llama 3 70B. Those are the company's own numbers, and it labelled them preliminary at the time. I would treat them as a direction, not a guarantee.

Gemini described a deal that was already signed

This was the most interesting part of Gemini's answer to me.

On 10 September 2026, d-Matrix announced a multi-year collaboration with Nvidia. It joined NVLink Fusion, the programme Nvidia opened in May 2025 that lets other companies' chips plug into Nvidia's high-speed rack connections. The plan is exactly what Gemini described. Nvidia's next GPUs, Vera Rubin, do the prefill. d-Matrix's next chip, called Raptor, does the decode. Both sit in the same Nvidia MGX rack design.

Raptor is a big change from Corsair. It stacks a DRAM chip on top of the SRAM compute chip in one package, which d-Matrix describes as a two-storey design. The point is to fix the capacity problem I showed in the second chart. Raptor is supposed to tape out before the end of this year, with the first systems inside Nvidia racks expected in the fourth quarter of 2027. So the most important chip in this story does not exist in silicon yet.

Then things moved quickly. On 9 October, The Information reported that Nvidia plans to invest in d-Matrix, citing three people familiar with the deal. Neither company had confirmed it. On 12 October, d-Matrix showed something called Squadron at the OCP Global Summit, a cluster of up to 1,024 Corsair cards across 16 air-cooled racks. In the same demo it paired Corsair racks with Nvidia H100 servers and reported 845 tokens per second for each user. In other words, the split is already being shown with today's chips, not just promised for 2027.

Nvidia is not fighting the challengers. It is collecting them.

Look at that table again and a pattern jumps out. Groq built fast, SRAM-heavy inference chips and was pitched for years as an Nvidia challenger. Nvidia paid for a licence and hired its founder. d-Matrix built fast, memory-centric inference chips and was pitched as an Nvidia challenger. Nvidia invited it into its own rack and, if the report is right, is about to own a piece of it.

An analysis from Futurum Group asked whether d-Matrix is Nvidia's hedge on Groq. I think that is the right question. The two seem to fit different sizes of model. Groq's approach is extremely fast but holds less, so it suits smaller models. Raptor is being aimed at very large ones. With both, Nvidia can offer a fast decode option for almost any customer, and it never has to give up the rack.

That is the bit I keep coming back to. The real prize is not the chip. It is the rack, the cables, the software and the customer relationship. Nvidia has worked out that it does not need to win every part of inference. It only needs every part of inference to happen inside a system that it designed.

So is d-Matrix a rival or a partner? My honest answer is that it started as a rival and is becoming a component. That is not a bad outcome for d-Matrix. Its funding talks have reportedly moved from a 2 billion dollar valuation towards 5 billion. But anyone hoping for a small startup to dethrone Nvidia should notice how this keeps ending.

As for Gemini, I give it a fair mark. It explained the technology clearly, and it saw the partnership coming before I did. It also doubled the one number that the whole argument rests on, and it missed that its "could happen" had already happened. I would not have known either thing if I had not checked. It felt a lot like my last long chat with Gemini, where the confident answer also turned out to be only half the story.

FAQ