Skip to main content

lemmy - Link to source

China open-source AI models surpass 10 billion downloads

China open-source AI models surpass 10 billion downloads

China's domestically developed open-source large language models have recorded more than 10 billion cumulative downloads worldwide, and the country now holds
admin (Daily Ittehad)
#technology
in reply to ☆ Yσɠƚԋσʂ ☆

neon_nova
lemmy - Link to source
neon_nova
• •

I have a 16gb MacBook Air m4.

I like the idea of having a model I can run locally in the event of a possible long term internet outage.

Can you recommend a model that would be suitable for my computer?

in reply to neon_nova

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •

16gb is a bit low unfortunately. You could run a 2 bit quant of latest Qwen, but that's going to be a severely degraded performance. huggingface.co/unsloth/Qwen3.6…

Might be worth trying though to see if it does what you need.


unsloth/Qwen3.6-35B-A3B-GGUF · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
in reply to ☆ Yσɠƚԋσʂ ☆

neon_nova
lemmy - Link to source
neon_nova
• •
Thanks! I figured it’s low on ram, but with the way things are going in the world, maybe it’s better than nothing is what I’m thinking.
in reply to neon_nova

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •

It's entirely possible we might see fairly capable models that can be run with 16 gigs of RAM in the near future. Qwen 3.5 came out in February, and you needed a server with hundreds of gigs of memory to run a 397bln param model. Fast forward to a couple of weeks ago and 3.6 comes out with a 27bln param version beating the old 397bln param one in every way. Just stop and think about how phenomenal that is qwen.ai/blog?id=qwen3.6-27b

So, it's entirely possible people will find ways to optimize this stuff even further this year or the next, and we'll get an even smaller model that's more capable.

Qwen Studio

Qwen Studio offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.
qwen.ai
in reply to ☆ Yσɠƚԋσʂ ☆

neon_nova
lemmy - Link to source
neon_nova
• •
Thanks! That’s really amazing to hear. I guess I’ll wait a bit and see what happens.
in reply to ☆ Yσɠƚԋσʂ ☆

Avid Amoeba
lemmy - Link to source
Avid Amoeba
• •
Still worth using Qwen3-Coder-Next 80B? Runs about slightly faster than 3.6 27B on my hw.
This entry was edited (3 weeks ago)
in reply to Avid Amoeba

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
I haven't tried comparing them myself, I guess you just kind of have to gauge if it works well enough. :)
in reply to ☆ Yσɠƚԋσʂ ☆

Avid Amoeba
lemmy - Link to source
Avid Amoeba
• •
What software are u using with the models for code? OpenCode, Nanocoder, etc.?
in reply to Avid Amoeba

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
I ended up settling on opencode, but I find all of them work more or less the same nowadays. Pi is an interesting one which is very minimalist.
in reply to Avid Amoeba

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
I've stopped bothering using an editor with LLMs. I just get the model to make a phased plan, write using TDD, and tell it to do staged commits for each feature. Then I just review the diffs after.
in reply to ☆ Yσɠƚԋσʂ ☆

Avid Amoeba
lemmy - Link to source
Avid Amoeba
• •
Interesting. And for web search u use the built-in or hook it up to SearXNG?
in reply to Avid Amoeba

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
I've just been using the builtin, but searxng might be better. Seems like a lot of people prefer it.
in reply to Avid Amoeba

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
I think so yeah, searxng is definitely the most privacy focused option.
in reply to ☆ Yσɠƚԋσʂ ☆

Avid Amoeba
lemmy - Link to source
Avid Amoeba
• •
What model do you use and on what hw? I recently got a R9700 to experiment with the various Qwen 3.5/3.6 models.
in reply to neon_nova

bountygiver [any]
lemmy - Link to source
bountygiver [any]
• •
long term internet outage is not that likely. But getting priced out of any online models is quickly the reality.
in reply to ☆ Yσɠƚԋσʂ ☆

AlHouthi4President
lemmy - Link to source
AlHouthi4President
• •
What is the difference between running locally and running on Qwen platform?
in reply to AlHouthi4President

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
Mainly data sovereignty. Running a local model means all your data stays on your machine. Any time you use a service you're sending whatever the model is working on to the company. Another advantage is the price. With services you have to pay a subscription, with local models you get to run them for the price of electricity.
This entry was edited (3 weeks ago)
in reply to ☆ Yσɠƚԋσʂ ☆

OldQWERTYbastard
lemmy - Link to source
OldQWERTYbastard
• •
The land of the CCP is the last place I'd expect to see FOSS AI agents. Good for them! Beats the hell out of our greedy bastards in the United States.
in reply to ☆ Yσɠƚԋσʂ ☆

Avid Amoeba
lemmy - Link to source
Avid Amoeba
• •
Yeah, I got a superbly functional and super fast search / research / assistant tool from Qwen 3.6 35B and Open Web UI + SearXNG. All running local. It passed the WAF benchmark with flying colors.
This entry was edited (3 weeks ago)
in reply to Avid Amoeba

☆ Yσɠƚԋσʂ ☆
lemmy - Link to source
☆ Yσɠƚԋσʂ ☆
• •
It's honestly incredible how good the local stack is nowadays. It's literally better than any frontier model you could've rented like a year ago.
in reply to comfy

qwerty
lemmy - Link to source
qwerty
• •
It's so good that some people download two in case the first one breaks.
⇧