Skip to main content

lemmy - Link to source

Global downloads of China's open-source AI models exceed 10 billion

Global downloads of China's open-source AI models exceed 10 billion: official

China’s innovation capability continues to strengthen since the beginning of 2026, with the penetration rate of AI technology across the country’s major large and medium-sized enterprises exceeding 30 percent, and the cumulative global downloads of C…
www.globaltimes.cn
#technology
in reply to ☆ Yσɠƚԋσʂ ☆

i_am_not_a_robot
lemmy - Link to source
i_am_not_a_robot
• •
Who is downloading these? I thought I would try out the new GLM before it becomes illegal, but these new models require 256GB RAM to run even the scaled down versions. I thought my PC I bought just before hardware prices went crazy was excessive but it only has 128GB. You're looking at spending over 5000 dollars on a PC that can just barely fit one AI model and only until they make new ones that need 384GB.
in reply to i_am_not_a_robot

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
There are smaller models like Qwen 3.6 27b that can run with as low ast 16gb VRAM. The larger models are largely used either by companies running them on prem, or research institutions.
in reply to i_am_not_a_robot

brucethemoose
lemmy - Link to source
brucethemoose
• •

You can run MiMo 2.5 in 128GB, as 3 bit quant. The model itself is less that 128GB as an IQ3_KT, and it’s KLD (measured quantization loss) is very reasonable.

I get about 9 tokens/sec on 128GB with a 7800 CPU and one RTX 3090. Intend to try dflash to speed it up this week. I can run up to 90k context, without too much kv quantization, depending on how I configure my PC.

And it’s a really good model, even quantized.

This entry was edited (4 days ago)
in reply to ☆ Yσɠƚԋσʂ ☆

userentity
lemmy - Link to source
userentity
• •

Qwen 3.6 27B MTP (Q4) is really great. ~24GB VRAM usage with one slot @ 262k . ~30GB with 3 slots at 128k. Also it does not struggle like Gemma if you use e.g. a Q4 KV-Cache. And it runs at 400-800 ppt/s and 20-60 itp/s on a V100.

But it has a competitor since a short while that is Laguna XS 2.1, which is really good for a A3B MoE. I'd never have thought a 30B MoE could be on par with a 27B dense, but it seemingly is.

⇧