An idea for #Fedilab: a feature to generate a description for your photos to help accessibility. You post an image, and a suggested alt text comes back for you to review and edit before publishing. It would run on a self-hosted open source vision model (#MoonDream: github.com/vikhyat/moondream) on my own server, in a closed loop with no cloud, no third-party service and no image stored. Would this be useful to you?
GitHub - m87-labs/moondream: tiny vision language model
tiny vision language model. Contribute to m87-labs/moondream development by creating an account on GitHub.GitHub
- Yes (58%, 115 votes)
- No (41%, 82 votes)
Kinky Kobolds
in reply to Fedilab Apps • • •Fedilab Apps
in reply to Kinky Kobolds • • •I perfectly understand. But LibreTranslate is maybe one of the most ethical translation services (used by Fedilab with its own server), and it was still built through AI, with pretrained neural models. The real question is not the technology itself, it is where your data goes. Here nothing leaks: a local model, with my own server, and no cloud, no third party. I would never propose it otherwise.
wortezimmer
in reply to Fedilab Apps • • •Sounds like you decided already.
@KinkyKobolds
Rémi Letot
in reply to Fedilab Apps • • •@KinkyKobolds
fedicat
in reply to Fedilab Apps • • •Florian
in reply to Fedilab Apps • • •don't we have this already with @altbot ? The used model differs, but apart from that it reads very similar.
#Fedilab #MoonDream
Mads Damgaard Mortensen (Å)
in reply to Fedilab Apps • • •thefathippy
in reply to Fedilab Apps • • •Detun3d
in reply to Fedilab Apps • • •I know vision models can be pretty light so that doesn't worry me much, and a few Mastodon instance admins have been testing bots with these to add descriptive comments to image posts uploaded without an alt text.
Your approach would probably be the best one and people with certain conditions will likely be thankful, so go ahead if you can. 👍
Hé
in reply to Fedilab Apps • • •I've never used a generator to describe an image, but I don't think it's a good idea.
When you translate a post with an automatic translator, you know the translation might be imperfect or even contain errors.
Here, the goal is to automatically provide an image description to people who are visually impaired or blind, without worrying about the quality of the description and without giving them a choice. I don’t think users will check and correct the
Jeder
in reply to Fedilab Apps • • •I think the idea itself is a good one. It should be possible to switch the function on or off as required.
#Fedilab #MoonDream
Wolkensteine 🏳️🌈
in reply to Fedilab Apps • • •Personally, I'd not use it, but I can see people benefiting from it.
I am quite concerned about genAI mostly and would love to see Fedilab to stay as open about such changes as now, even when that leads to people being mean … If there is a fully open-source and not just open-weight model that would be even better but I am not sure if that exists for this usecase; that would address concerns in terms of stealing from people, as the data is public to review.
Thank you so much for creating free software
nicolaottomano
in reply to Fedilab Apps • • •As I suggested in another post (the one for TTS):
1. I support the cloud-based AI ALT text generator, called directly into Fedilab
2. For devices with enough CPU/GPU power, I would go with on-device AI (Android has something which could be integrated into apps developer.android.com/ai/overv… )
The 2nd option should be seen as a way to relieve the server of its workload (or to have a smaller, cheaper, server)
Find the right AI/ML solution for your app
Android Developers