No more 32 GB consumer cards, the next step up is a "Pro" card with an extra digit on its MSRP.
I'm reminded of the "beginning of the end" for 3dfx where they kept doubling-down on basic scaling without fundamental improvements, and were wiped out by more efficient cards from upstart little manufacturers like... NVIDIA.
I'm going to rub my crystal ball and bet that a Chinese vendor will start making consumer cards with 256 GB of High Bandwidth Flash (HBF) and a smaller DRAM frame buffer, maybe just 8 GB or so. Flash memory would work fine for either texture storage or for AI weights, so such a card would sell like hotcakes!
Intel could have had an interesting play there since they have GPUs and they have the tech for a product between flash and DRAM performance.
Honestly, it seems like a perfectly plausible solution: With 20us latency, you'd be able to load 50k layers per second. You can get as much bandwidth as you need by putting more dies in parallel.
Qwen3.8-flash-next has 48x sparse-MoE layers, and you don't know which experts need to be loaded for each layer until the prior layer has finished. So that requires 48x 20us delays per forward pass. That admits more than 1000 forward passes per second.
With typical MTP acceptance rates, you'd be looking at maybe over 2k tok/sec as the theoretical upper bound. A big flash memory on the GPU would be great for n-gram tables as well.
The big difference now is that the fundamental shift in component pricing is making it hard for competitors to bring something with a significantly better bang for your buck factor. I strongly doubt we'd see these moves made by NVIDIA if component prices were "normal", I mean even PCBs are affected[1].
[1]: https://www.tomshardware.com/pc-components/ai-demand-trigger...
RTX 5090 Street Prices Top $9k as Nvidia GPU Costs Climb