Claude: papers please?
I'm a big proponent of self-hosting, right to repair, and rolling your own whatever when you can. That probably started as teenage rebellion that got baked in - I was lucky enough to read both Walden and The Hobbit during a week-long cyclone lockdown several decades ago - but I suspect there's a non-trivial overlap between that space and privacy-minded people in general.
My endgame is a self-sufficient intranet for myself and family: if the net goes down tomorrow, we'd barely notice.
I also use LLMs as a tool. True self-hosted equivalence to state-of-the-art models is still an expensive proposition, so like many, I use cloud-based tools like Claude or Codex for domain-specific heavy lifting - mostly coding. Not apologising for it; I think it's a reasonable trade-off while local hardware catches up.
That context is just to establish where I'm coming from when I say this caught my attention today:
support.claude.com/en/articles…
To be accurate about what it actually says: this isn't a blanket "show us your passport to use Claude." Not yet.
The policy as written is narrower than it might first appear.
My concern isn't what it says - it's that the precedent now exists. OAI will do doubt follow suite.
Scope creep is a documented pattern with this kind of thing, and "we only use it for X" describes current intent, not a structural constraint.
Given the nature of this community, figured it was worth flagging.
like this
giantpaper likes this.
Eager Eagle
in reply to SuspciousCarrot78 • • •like this
giantpaper likes this.
unitedwithme
in reply to Eager Eagle • • •SuspciousCarrot78
in reply to Eager Eagle • • •I'm not sure that solves the issue or just changes the actors. Still, I'm all for "fight the power".
I'm just a silly man with a box of scraps. But I hope enough silly men with boxes can come together to form some sort of co-op. Maybe. I don't know. But...I hope, people smarter and better resourced than I can find a way forward.
The writing is on the wall here.
Sims
in reply to SuspciousCarrot78 • • •trailee
in reply to SuspciousCarrot78 • • •SuspciousCarrot78
in reply to trailee • • •I hope you're wrong. I'm worried that you're probably not.
Still time. Just barely.
trailee
in reply to SuspciousCarrot78 • • •utopiah
in reply to SuspciousCarrot78 • • •IMHO LLM usage isn't coherent with independence. That being said I wrote quite a bit on self-hosting LLMs. There are quite a few tools available, like ollama itself relying on llama.cpp that can both work locally and provide an API compatible replacement to cloud services. As you suggested though typically at home one doesn't have the hardware, GPUs with 100+GB of VRAM, to run the state of the art. There is a middle ground though between full cloud, API key, closed source vs open source at home on low-end hardware : running STOA open models on cloud. It can be done on any cloud but it's much easier to start with dedicated hardware and tooling, for that HuggingFace is great but there are multiples.
TL;DR: closed cloud -> models on clouds -> self-hosted provide a better path to independence, including training.
SuspciousCarrot78
in reply to utopiah • • •Yeah, me too :)
bobbyllm.github.io/llama-condu…
codeberg.org/BobbyLLM/llama-co…
I'm thinking about coding a >>cloud side car at the moment, with the exact feature you mentioned...but...that's scope creep for what I have in mind.
Irrespective of all that, I agree: an open cloud co-op could be a good way to have SOTA (or near SOTA - GLM 5.1 is about as close as we have right now) access for when needed.
(Not teaching you to suck eggs, so this comment is for the lay-reader):
For coding, you can do some interesting stuff where the cloud model is the "general" and the locally hosted LLM is the "soldier" that does the grunt work. We have some pretty decent, consumer-level-hardware runnable "soldiers" now (I still like Qwen 3 coder)...they just don't quite have the brains to see the full/big picture for coding.
llama-conductor
Codeberg.orgsteel_for_humans
in reply to SuspciousCarrot78 • • •Say I have a GPU with 32GB VRAM and I am on Linux, what local LLM would be good for coding?
Currently I just have an iGPU ;) but that's always an option, albeit a very expensive one.
SuspciousCarrot78
in reply to steel_for_humans • • •Sadly...none. Well, I mean...it depends what you mean by "coding". If you mean "replace Claude with local?". Then...none. Sorry.
If you mean "actually, if I use ECA to call a cloud model from OpenRouter for planning, then have it direct a local LLM to do the scutt work", then the Qwen series of models (like Qwen 3 Next) are pretty awesome.
The iGPU will make you want to kill yourself though. Get a GPU :) Even a 4-16GB one can make a difference.
PS: You said GPU and iGPU, so I'm not sure which one has the 32GB or what rig your running. I have suspicion though you're running on a i5 or i7 with something like a intel 630 igpu inbuilt? In which case, the iGPU is pretty slow and depending on the exact chip, you likely won't be able to use CUDA or Vulkan acceleration.
So, the "get a GPU" thing still holds :)
ECA - Free Open Source AI Pair Programming in Any Editor
ECA - Editor Code Assistantsteel_for_humans
in reply to SuspciousCarrot78 • • •I meant that I can buy one of those Radeons dedicated to AI work, like the ASRock Radeon AI PRO R9700 Creator 32GB GDDR6. If I need to.
Currently my Ryzen iGPU is all I need, because all I need is to see the graphical desktop environment on my screen ;) It does the job well.
I use Claude Code as well and I am slightly concerned with that ID verification news, even more so because of the technology partner that they chose.
SuspciousCarrot78
in reply to steel_for_humans • • •Hmm. The R9700 is RDNA4 - ROCm support for that architecture may be patchy in linux? Dunno. Check that before you commit your hard earned dollary-doos.
If all good
Qwen 3.6 is the latest hotness. I'd start from there and work backwards
inv.nadeko.net/embed/YKNvkBbRJ…
- YouTube
www.youtube.comandrew0
in reply to steel_for_humans • • •Get llama.cpp and try Qwen3.6-35B-A3B. Just came out and looks good. You'll have to look into optimal settings, as it's a Mixture of Experts (MoE) model with only 3B parameters active. That means that the rest can stay in RAM for quick inference.
You could also try the dense model (Qwen3.5-27B), but that will be significantly slower. Put these in a coding harness like Oh-My-Pi, OpenCode, etc. and see how it fares for your tasks. Should be ok for small tasks, but don't expect Opus / Sonnet 4.6 quality, more like better than Haiku.
lsjw96kxs
in reply to SuspciousCarrot78 • • •SuspciousCarrot78
in reply to lsjw96kxs • • •No point in telling you "yo, dude, just grab MinMax 2.7 or GLM5.1"...unless you happen to have several GPUs running concurrently with a total combined VRAM pool of 500GB or more.
There are strong local contenders... (Like Qwen3-Coder-Next but as you can see, the table ante is probably in the 45GB vram range just to load them up. Actually running them with a decent context length is likely to mean you need to be in the 80-100GB range.
Do-able...but maybe pay $10 on OpenRouter first to test drive them before committing to $2000+ worth of hardware upgrades.
There are other, more reasonable, less hardware dependent uses for local LLMs, but if you want fully local coders, it's the same old story: pay to play (and that's even if you don't mind slow speed / overnight batch jobs).
Right now, cloud-based providers are hemorrhaging mo
... Show more...No point in telling you "yo, dude, just grab MinMax 2.7 or GLM5.1"...unless you happen to have several GPUs running concurrently with a total combined VRAM pool of 500GB or more.
There are strong local contenders... (Like Qwen3-Coder-Next but as you can see, the table ante is probably in the 45GB vram range just to load them up. Actually running them with a decent context length is likely to mean you need to be in the 80-100GB range.
Do-able...but maybe pay $10 on OpenRouter first to test drive them before committing to $2000+ worth of hardware upgrades.
There are other, more reasonable, less hardware dependent uses for local LLMs, but if you want fully local coders, it's the same old story: pay to play (and that's even if you don't mind slow speed / overnight batch jobs).
Right now, cloud-based providers are hemorrhaging money because they know it will lead to lock-in (ie: people will get use to what can be achieved with SOTA models, forgetting the multi-million dollar infrastructure required to run them). Then, when they realize you can't quite do the same with local gear (at least, without spending $$$), they can ratchet the prices up.
Codex pro-plan just went to $300/month.
We've seen this playbook before, right?
unsloth/Qwen3-Coder-Next-GGUF · Hugging Face
huggingface.colsjw96kxs
in reply to SuspciousCarrot78 • • •lime!
in reply to lsjw96kxs • • •lsjw96kxs
in reply to lime! • • •lime!
in reply to lsjw96kxs • • •ropatrick
in reply to SuspciousCarrot78 • • •I love the sound of this but can I ask, if the net goes down and you hardly notice, where do you get your 'net' from? Or is it that your intranet doesn't need internet as such and everything is just local?
I might have answered my own question there but I'm interested to understand it a bit more.
Thanks!
SuspciousCarrot78
in reply to ropatrick • • •The intranet becomes the internet :) Everything is local, accessible from multiple devices within my wLAN. The main box plugs into the router and serves everything over Wifi to trusted devices - my documents, media, books, games etc.
I wrote (flippantly) about the bones of the system here, 3 or 4 months ago. It's more complex now, but the endgame has always been "what if cloud, but you are your own cloud?"
lemmy.world/post/41315607/2143…
It may not be fresh (if the net goes down), but it would be local. The only real question I have to grok for myself is if I want to mirror curated section of Wikipedia, books etc.
en.wikipedia.org/wiki/Kiwix
Probably I should. May as well go full data-horder. Good excuse to get a few more TB of storage. What I've done so far is all within 4TB, using clever tricks and black magic but there's a limit to 4TB.
... Show more...The intranet becomes the internet :) Everything is local, accessible from multiple devices within my wLAN. The main box plugs into the router and serves everything over Wifi to trusted devices - my documents, media, books, games etc.
I wrote (flippantly) about the bones of the system here, 3 or 4 months ago. It's more complex now, but the endgame has always been "what if cloud, but you are your own cloud?"
lemmy.world/post/41315607/2143…
It may not be fresh (if the net goes down), but it would be local. The only real question I have to grok for myself is if I want to mirror curated section of Wikipedia, books etc.
en.wikipedia.org/wiki/Kiwix
Probably I should. May as well go full data-horder. Good excuse to get a few more TB of storage. What I've done so far is all within 4TB, using clever tricks and black magic but there's a limit to 4TB. Fortunately, hard drives are still not too $$$. +4TB is about $200 here locally. So the entire set up is still around $600-700 AUD (around $350-400 USD)
All the other stuff I have more or less tee-ed up (barring the UPS + solar kit I am building later in the year).
Anyhow, 8TB local should just about cover what I have in mind (he said, fully aware of dragon horder sickness). Then I'll grab something for offsite storage for critical docs - I have an old raspberry pi with a 256GB NVMe ssd I can use for that.
I'm semi tempted (because fuck it, why not have fun) to look into LoRA after that.
en.wikipedia.org/wiki/LoRa
What I really NEED to do is finish the LLM stack (I'm on it and nearly done) and then do a curated Youtube replacement with yl-dlp feeding into Nova-player or Jellyfin, once/if SmartTube etc gets shit-canned. The youtube thing I'm kinda excited about because I've figured out how to squeeze ~1500 videos in around 250ish GB of storage, with TTL (time to live) mechanics, download replacement schedule etc. The kids watch too much random shit on YT, so daddy will make YT at home (ha!).
I have some other wild ideas too...it's a whole other thing...don't get me started :)
Once I'm finished, I will open-source the entire thing, post about it here, and let others replicate / improve on it. And so it goes. Once you begin walking down the dark path, you are forever doomed. Be careful :)
- YouTube
www.youtube.comropatrick
in reply to SuspciousCarrot78 • • •OK I have you. You dont need the internet because you have the internet in your terabyte farm. Pretty cool.
Thanks for the detailed reply.
One final question, I'm sure its dark at the bottom of the deep rabbit hole you are in, what do you do for batteries for your head torch?! 😀
SuspciousCarrot78
in reply to ropatrick • • •Exactly so. Mom - can we get the internet? Mom: we have the internet at home.
Batteries? I don't need batteries. I have the never-ending warm glow of weaponized autism. And that's not even a joke.
I tend to hyper-fixate on something until either it breaks or I do. It's usually 70/30 in my favour :)
llama-conductor/DESIGN.md at main
Codeberg.orgwebghost0101
in reply to SuspciousCarrot78 • • •SuspciousCarrot78
in reply to webghost0101 • • •webghost0101
in reply to SuspciousCarrot78 • • •Yep, definitely talking to myself again
Jokes aside, i haven’t seen you mention anything for media streaming.
I highly passionately recommend Navidrome for music. It is my absolute favorite and most used self hosted service.
For acquiring media like film and music depending where you live ripping those from your local library is in some places arguably a protected fair use. (Comes from the time mp3
players became common and runners used to take rented cds in their walkman outside before). In my experience, 480p dvd is much higher quality then internet 480p streams and the total size is much smaller then what you find in downloads.
ARM can help you automatically rip these as long as you have a drive in your pc. I got ARM running in a proxmox lxc with drive passtrough but that honestly was a pain to setup so not sure you should go that exact route, either way the moment arm is functional its smooth sailing and your only concern becomes storage space.
GitHub - automatic-ripping-machine/automatic-ripping-machine: Automatic Ripping Machine (ARM) Scripts
GitHubSuspciousCarrot78
in reply to webghost0101 • • •Well, this is going to freak you out, because I am (literally, right now) explicitly scoping out offline YouTube integration into Jellyfin, as a sort of rolling library. Jellyfin has been good to me, but I've been using Nova Player for a while now, since my Pi borked itself (Nova player is plug hard drive into router, install app on TVs, done). The limit is that yt-dlp doesn't integrate very well with it. I mean, I could build something, or fork the repo myself...or I could just use what already exists.
So it might be time to restore the entire *arr stack.
The TL;DR: I want one front end for ALL my media - YouTube, instructionals, movies, TV shows. That immediately speaks to Jellyfin, which I'm very familiar with. The issue is YouTube. There's too much slop on there, I want a curated experience for the kids, SmartTube won't work forever, and the eldest is starting to go black-hat and screw around with settings. That's accelerating the timeline.
The stack I'm scoping:
- Jellyfin - front end for everything
- Tube Archivist
... Show more...Well, this is going to freak you out, because I am (literally, right now) explicitly scoping out offline YouTube integration into Jellyfin, as a sort of rolling library. Jellyfin has been good to me, but I've been using Nova Player for a while now, since my Pi borked itself (Nova player is plug hard drive into router, install app on TVs, done). The limit is that yt-dlp doesn't integrate very well with it. I mean, I could build something, or fork the repo myself...or I could just use what already exists.
So it might be time to restore the entire *arr stack.
The TL;DR: I want one front end for ALL my media - YouTube, instructionals, movies, TV shows. That immediately speaks to Jellyfin, which I'm very familiar with. The issue is YouTube. There's too much slop on there, I want a curated experience for the kids, SmartTube won't work forever, and the eldest is starting to go black-hat and screw around with settings. That's accelerating the timeline.
The stack I'm scoping:
The YT stack: rolling library logic:
Scoping the maths at 200GB, 30-min average per vid, using compressed modern codecs:
Planning numbers per video: assume average video is 30mins. At 360p, that's ~100MB per video. 480p ~160MB, 540p ~220MB, 720p ~320MB.
If I have a selection of "core keepers" at 720p H.265 (~300 videos), taking up ~80GB, that leaves ~120GB for the rolling pool:
I don't need 4K...hell, 1080p is wasted on me. So I'm thinking.... 300 core vids at 720p + rolling library at 540p = 845 videos, give or take. More than enough to keep the fam off my back once SmartTube goes tits up (they can't play whack-a-mole for ever).
I would prefer a clean migration to other, live sources (I have those scoped out as well) but not all the Minecraft / gaming / pretend play / blah blah stuff the family watches is on Peertube/Odysee/Curiosity Stream.
PS: I see your 480p and raise you 60, because 540p is the forbidden resolution :)
PPS: I was planning on using JF for music too...but maybe I should look at Navidrome like you said.
The crazy idea that I had was to use AI to create an infinite playlist of sorts. Seed it with your own music, get it to generate tracks in THAT style as filler, intermingle them (so there's always something new).
Finish off with AI DJ's that pulls in "local news" from your curated RSS feeds.
Think: Three Dog from Fallout 3.
Basically what I spoke about here -
lemmy.world/post/43936980/2278…
I have a pretty clear idea of how to get that done. It could be amusing.
huggingface.co/ACE-Step/aceste…
ACE-Step/acestep-5Hz-lm-0.6B · Hugging Face
huggingface.cowebghost0101
in reply to SuspciousCarrot78 • • •Currently my setup for YouTube In jellyfin is pretty basic (but it works well)
Its one complex yt-dlp script that runs ecery hour, reads a list of channels from a txt file. Rss to check the last 2 vids per channel, if there not downloaded already (log with vid ids per channel) it starts to download on high quality preset, filter out shorts and “live”, if the result is getting above a certain size it aborts and tries again with lower quality preset.
I don’t have a system to auto dele yet cause the script is only for my favorites. For everything else or that uploads to regularly i host a local invidious.
But google really hates invidious and it breaks frequently, even got my original residential ip banned. I may take a look at tube archive cause i would prefer download and delete as a more stable flow.
Something else i am experimenting with is using the same data in multiple systems. My music collection is a proxmox dataset but as long as the read rights are good i see no reason why jellyfin can’t also read those files.
The outpu
... Show more...Currently my setup for YouTube In jellyfin is pretty basic (but it works well)
Its one complex yt-dlp script that runs ecery hour, reads a list of channels from a txt file. Rss to check the last 2 vids per channel, if there not downloaded already (log with vid ids per channel) it starts to download on high quality preset, filter out shorts and “live”, if the result is getting above a certain size it aborts and tries again with lower quality preset.
I don’t have a system to auto dele yet cause the script is only for my favorites. For everything else or that uploads to regularly i host a local invidious.
But google really hates invidious and it breaks frequently, even got my original residential ip banned. I may take a look at tube archive cause i would prefer download and delete as a more stable flow.
Something else i am experimenting with is using the same data in multiple systems. My music collection is a proxmox dataset but as long as the read rights are good i see no reason why jellyfin can’t also read those files.
The output folder of ARM used to be its own library in jellyfin on my first server.
SuspciousCarrot78
in reply to ropatrick • • •Here. Begin your walk down the path to a power some would call...unnatural
get.kiwix.org/en/solutions/app…
get.kiwix.org/en/solutions/hot…
get.kiwix.org/en/solutions/hot…
Kiwix - Kiwix Server
Kiwixmaegul (he/they)
in reply to SuspciousCarrot78 • • •Probably a shallow response …
But I always figured AI/LLMs are basically apocalyptic for all sorts of individualistic values in computing (including privacy but also independence and diversity).
Whether they’re good or useful etc, I just struggle to see how they will ever be justifiable against these sorts of values.
Sure, local models and our hardware will get better … but better than the state of the art from the big labs and providers? Given that data and training are the big bottlenecks on quality … I struggle to see how AI isn’t a complete feudal capturing of information computing and processing. Not to mention what happens to the pipeline that produces information content if everyone is only consuming it through the models that train on it.
So for me the big question is, what’s our call on a possible (likely even?) future where we are forever stuck using cloud provided AI along with all of its negatives, in the same way that basically all of us has been and still is stuck using MS windows, Google and the big-soci
... Show more...Probably a shallow response …
But I always figured AI/LLMs are basically apocalyptic for all sorts of individualistic values in computing (including privacy but also independence and diversity).
Whether they’re good or useful etc, I just struggle to see how they will ever be justifiable against these sorts of values.
Sure, local models and our hardware will get better … but better than the state of the art from the big labs and providers? Given that data and training are the big bottlenecks on quality … I struggle to see how AI isn’t a complete feudal capturing of information computing and processing. Not to mention what happens to the pipeline that produces information content if everyone is only consuming it through the models that train on it.
So for me the big question is, what’s our call on a possible (likely even?) future where we are forever stuck using cloud provided AI along with all of its negatives, in the same way that basically all of us has been and still is stuck using MS windows, Google and the big-social-media hellscape?
For me, I baulk at this.
SuspciousCarrot78
in reply to maegul (he/they) • • •I'm right there with you...but may I offer an alternative narrative in two parts and then address the pipeline issue you raise.
The first part:
There's a small (but real) subset of people turning their back on big corpo. Retro-tech, dumb-phones, self hosting, linux, right-to-repair advocates, OSS and FOSS, privacy groups .... everyone can smell the enshittification and are (in their own ways) pushing back. That's not nothing.
I think the way forward is not to play the game. Big corpo will do what big corpo always does. But we can use the tools we have to make the things we want.
Will it compete with SOTA? No. But...does it need to? At an individual level, I'd argue "probably not". It just needs to work for the individual.
More to the point, there's something to be said about doing more with less. Constraints can bring about real innovation. If the answer cannot be "Throw more X at it" (where X is $$$, compute, whatever)...then how can you leverage the tools and intelligence you have to build what you want? I think that's the real question.
Now f
... Show more...I'm right there with you...but may I offer an alternative narrative in two parts and then address the pipeline issue you raise.
The first part:
There's a small (but real) subset of people turning their back on big corpo. Retro-tech, dumb-phones, self hosting, linux, right-to-repair advocates, OSS and FOSS, privacy groups .... everyone can smell the enshittification and are (in their own ways) pushing back. That's not nothing.
I think the way forward is not to play the game. Big corpo will do what big corpo always does. But we can use the tools we have to make the things we want.
Will it compete with SOTA? No. But...does it need to? At an individual level, I'd argue "probably not". It just needs to work for the individual.
More to the point, there's something to be said about doing more with less. Constraints can bring about real innovation. If the answer cannot be "Throw more X at it" (where X is $$$, compute, whatever)...then how can you leverage the tools and intelligence you have to build what you want? I think that's the real question.
Now for the second part:
I'm more sanguine about it because I think this is down to the individual. Look at where you are now - it's not Reddit or Facebook :). You and I choose to be here because...reasons. We can choose to run Linux, LibreOffice, Mullivad, llama.cpp, SearXNG, Syncthing, Immich etc for the same reasons.
I think the trick will be figuring out how to navigate from your home ecosystem into the wider world, without getting f'd in the a.
The one thing I don't have a clean answer for is your pipeline point. If the content web collapses into AI slop - and it's already going that way - then the human-generated signal that makes these models worth using starts to degrade. You may need to hold onto your "Good Old LLMs" for a while yet (or start training your own from scratch. There are ways and means but that's beyond the scope of this conversation I think).
In any case, individual sovereignty doesn't fix that. You can opt out personally and still live in a world where the epistemic commons has been strip-mined.
That...probably what WILL happen, come to think of it. Ok, fine. But partial answers already exist - cryptographic provenance of human content, federated communities being structurally harder to slop-flood (maybe).
Honestly? Nobody has solved that problem just yet. The people building the biggest models know it's a problem and don't have a clean answer either. Anyone who says they do is selling something.
All I can say is the only way to win is not to play the game. Which WORP would no doubt meep-morp at.
maegul (he/they)
in reply to SuspciousCarrot78 • • •Oh I hear you (and appreciate the response).
For me, I can’t help but think of another alternative, which I’m surprised I haven’t heard of yet …
stripping down one’s personal technological cognitive load to a stack of systems that can fit into one’s brain (like the Python mantra), focusing on learning that stack well building sustainable and stable systems, and then just detoxing from the increasingly polluted digital information stream (protected commons, traditional formats such as books and in person engagement … dunno).
Depends on what the end goal is, but AIs seem to be about using tech more or just opting out of sovereignty. Something like the above seems to me to be about using tech less (in the end) and pushing toward being a secondary tool rather than an end of its own.
SuspciousCarrot78
in reply to maegul (he/they) • • •I agree.
God help me, I'm actually reading books again.
Books.
It's...harder than it use to be. A lot harder, actually.
But there's something to be said about marginalia etc.
maegul (he/they)
in reply to SuspciousCarrot78 • • •☂️-
in reply to maegul (he/they) • • •pound_heap
in reply to SuspciousCarrot78 • • •The fuck it does. Claude is already censored - you can't get a recipe of a poison, schematics for a bomb, an advice on how to hide a body. If you can, then it's Anthropic engineers didn't do their job.
Knowing who is using it helps either with conditional censorship, or helping governments to track people based on their prompts, or just plainly lying and using data for analytics and training. All these easons are shit.
And don't tell me this is to protect the kids again. Let the parents do their job.