Skip to main content

File storage and bandwidth. These expenses will also only get more expensive over time. File bandwidth is the #1 charge on the monthly server bill right now. I live in fear of an AI scraper figuring out how to scrape all of these files and bankrupting me overnight.


from RIP botsin.space.

I would expect a "fastly will solve all your problems" from this and not an attempt to write an optimized codebase. But that's just me.

Now for the advertisement (thought you were safe from it here). My benchmarking at data.funfedi.dev/ doesn't indicate a performance degradation in Mastodon v4.3.1. Of course, it's the receiving messages use case, and not the sending and offering up downloads. So it's completely irrelevant.

Now Helge out.

Interoperability Data for the Fediverse

data.funfedi.dev
#1
in reply to Helge

Helge
Helge
• •
Let's fix media storage in the Fediverse. There's no reason that the almost 30 thousands servers all have an AWS S3 account where they mirror all cute pictures posted on botsin.space.
in reply to Helge

bumblefudge
bumblefudge
• •
why not just use IPFS-or-equivalent to sync content-addressed media storage? original hosting server can "pin" (i.e. persist until user asks them to delete, whether using an IPFS-style DHT or just regular HTTP distro), relays cache, individual servers can just fetch live and not cache at all. content-addressing solves dedup, trusted relays can cache and be allowlisted/CORS-policied, etc.
codeberg.org/fediverse/fep/src…

fep/fep/cd47/fep-cd47.md at main

fep - Fediverse Enhancement Proposals
Codeberg.org
in reply to bumblefudge

bumblefudge
bumblefudge
• •
(full disclosure, i work for the IPFS Foundation, but i'm happy to outline the pro's and con's of "rolling one's own" and using equivalent technologies rather than the IPFS bundlings thereof. i just use "IPFS-or-equivalent" in my explanations cuz lots of people know how IPFS works, approximately.)
in reply to bumblefudge

Helge
Helge
• •
@helge why not just use


This has a simple answer: "I don't know".

However, I expect the engineering effort to adapt Mastodon and its clients to be able to make use of IPFS to be substantial. So this is not something that can be used for short term relief.

The ideal solution would be something like "Update to Mastodon 4.4.0 and add these three config variables". I don't see something like that involving new technologies.

in reply to Helge

Emelia 👸🏻
Emelia 👸🏻
• •

Jortage comes to mind, but I think it'd also be possible to instruct software "hey, I trust this server not to track me, serve me their media directly"

I know @shlee has thoughts here

@:PUA: Shlee fucked around and
in reply to Emelia 👸🏻

:PUA: Shlee fucked around and
:PUA: Shlee fucked around and
• •

I've messaged the Jortage dev to see if I can throw money at them to build a Jortage 2.0 (deployable edition)... they haven't replied

my main rant: shlee.fedipress.au/2024/call-t…


Call to Action: Fediverse Media Server

Edit: So to put my money where my mouth is - Right now storage is costing me around $150AUD a month - so if somebody can build a media server we can share/deploy and offer it for all of the Fediverse instances...
Shlee
in reply to :PUA: Shlee fucked around and

bumblefudge
bumblefudge
• •
if they don't want to DO the work but are ok "blessing" the work/fork, i would recommend rolling this up into a more full-featured Production-Grade Relay type project. I think such a Relay could charge less per month than the hosting fees it saves its servers, for a certain sweet spot of traffic/usercount... not exactly bankable in the VC sense but maybe a sideline that could help an already-trusted resource pooler like IFTAS?
in reply to bumblefudge

:PUA: Shlee fucked around and
:PUA: Shlee fucked around and
• •

I've been singing the praises of shared services in the fediverse for a long time.

I think IFTAS has their hands full.. but I also think one or two people with domain specific experience could get this done... if I knew the right people I'd ask them :annoyingdog:

This entry was edited (3 weeks ago)
in reply to :PUA: Shlee fucked around and

Emelia 👸🏻
Emelia 👸🏻
• •
I don't think IFTAS would be the right host for a project like this, but maybe Social Web Foundation or Fedihosting from @ruud would be good hosts?
@Ruud
in reply to Emelia 👸🏻

:PUA: Shlee fucked around and
:PUA: Shlee fucked around and
• •

I'm moving to better colo soon with 10x bandwidth....

so I could host a regional node once that's done (still need to write the server itself)

in reply to bumblefudge

Helge
Helge
• •

@shlee and al. I've added some relevant sequence diagrams to my fairer federation draft fep. Main point is the media server case, which has two consequences:

  • Less bandwidth used by the posting server
  • Less storage used by distributing servers

Both are good. Is this the picture everybody has in mind?


fep/fep/0008/fep-0008.md at drafts

fep - Fediverse Enhancement Proposals
Codeberg.org
in reply to Helge

:PUA: Shlee fucked around and
:PUA: Shlee fucked around and
• •

I don't understand the protocol enough so please take this toot as a note.

My objective is closer to just offering a S3 compatible service and adding the AP smarts to that in the backend (relay style to push/pull media between media servers).

but having a AP level support for media server location makes sense (maybe falling back to local if the mediaserver goes down or gets replaced) or even supporting a primary/second media server on the instance level in case of failure.

also, I'd like to see the AP include a UUID/hash of every media file so that in theory, instances/clients can migrate/roam from media server to media server or new media servers can easily find missing files.

but in principle yes :) Thank you for your efforts.

edit: I'm interested in the options around per toot media location. Because right now, the media URL is hardcoded on a toot by toot basis, and changing that media path retroactively feels hard/impossible. see github.com/mastodon/mastodon/p…


Add maintenance script to fix remote_url when media server changes by noellabo · Pull Request #16414 · mastodon/mastodon

Object storage may be unavoidably changed on the remote server in operation. This will invalidate the link to the original media because the remote_url of the media already obtained will be that of...
GitHub
This entry was edited (3 weeks ago)
in reply to :PUA: Shlee fucked around and

bumblefudge
bumblefudge
• •

> also, I'd like to see the AP include a UUID/hash of every media file so that in theory, instances/clients can migrate/roam from media server to media server or new media servers can easily find missing files.

This would be a major change to/extension of the protocol, BUT I completely agree that it would be worth exploring. Hash-addressing opens up multiple "fallback" methods to implementations that want to "heal" broken paths...

in reply to bumblefudge

bumblefudge
bumblefudge
• •

also, as far as making the *protocol itself* smarter about the expensive and clunky nature of uploads, it's worth mentioning that the spec itself has extremely little to say on the subject, and it would seem the authors of the protocol spec *assumed ongoing work would be done by the CG at the implementation/software level* : w3.org/TR/activitypub/#uploadi…

We could... spin that CG work item back up? it feels like a placeholder that Rhiaro made 3 commits to...

ActivityPub

The ActivityPub protocol is a decentralized social networking protocol based upon the [ActivityStreams] 2.0 data format.
www.w3.org
in reply to bumblefudge

Helge
Helge
• •

Well, I guess one needs to explain more than I've done so far. I'll try to update fairer federation.

Also this is really not an AP thing. I also believe that it is imperative to decouple the solution here. It's about sharing the work load in hosting media. That's relevant to every decentralized network ...

in reply to :PUA: Shlee fucked around and

Helge
Helge
• •

After thinking and self reflecting some more, I have two things to say:

  • There is a business case: If the costs are 330 $ / month for bandwidth and 70 $ /month for storage and one one can half storage, and passes half the savings along to the instance owners. One makes 20 $ / month profit per instance. So if one has like 500 instances using one's services one makes quite a bit of cash.
  • If my economics is right this means for 380$ in revenue one makes 20$. That's about 5% profit. I think that's okish for a company. But there's taxes, etc ...
  • Also with 500 instances, that's 190k$ a month of bills to pay. That's not a small business.

So what one would really need for this is a founder that knows if this is a "serious business case". I have no idea how to convince a bank to give me an account that can handle that kind of volume. I have no idea, how I would collect the bills. I have no idea how I would fairly split the bills between instances.

If you are a founder, and need someone to estimate the coding costs for this, and do the actual work, please feel free to reach out to me.

⇧