Skip to main content

mastodon - Link to source

Good morning Fedi friends!

I had to delete the toot I published earlier because sharing a URL to my new self-hosted Ghost blog overwhelmed the server & took the whole site down (even with powerful caching in place).

This is apparently a common problem: github.com/mastodon/mastodon/i…

Anyway! I spent the past few months learning some basic coding & I was able to manually install Ghost, edit Nginx configs & do other dev magic. It's super empowering! 💪

As they say: link in bio 😅

#MySoCalledSudoLife

a graphic with the title "the future is federated" set against the backdrop of a galaxy. below the title you see a URL with the subdomain "blog" erased by hand... pointing to a new URL that reads news.elenarossini.com

Reduce load of preview fetching on third-party servers · Issue #23662 · mastodon/mastodon

Steps to reproduce the problem Posting an image on Mastodon causes hundreds or thousands of federated servers to fetch the preview details all at once, causing a high load for the target server and...
GitHub
#mysocalledsudolife
in reply to Elena Rossini ⁂

Troed Sångberg
mastodon - Link to source
Troed Sångberg
• •

ping @renchap - I know this is on the roadmap but there seems to be different opinions on how this works considering the 0-60 sec random delay in Mastodon before fetching. Here's a recent case.

@_elena

@Elena Rossini ⁂ @renchap (not main account)
in reply to Troed Sångberg

Renaud Chaput
mastodon - Link to source
Renaud Chaput
• •
@troed @_elena I highly doubt that this is what happened here, especially if there is "powerful caching" in place. If your website with caching can not handle a few thousand requests over 60s, there is a problem somewhere else.
This is still on our roadmap, but I need to point out that this is not a Mastodon issue but a Fediverse-wide one, and despite many people pointing our to the issue, nobody wrote an FEP (or started the discussions to do one). We will do it, when we have some bandwidth…
@Troed Sångberg
in reply to Renaud Chaput

Troed Sångberg
mastodon - Link to source
Troed Sångberg
• •

@renchap

Thanks for the reply, yes, this is exactly why I pinged you. We have those who've spent a lot of developer resources on trying to sort it out their end (CoffeGeek) and others like Elena here who might not have - but the experience is still the same, creators have their web sites completely downed for several minutes after posting to Mastodon.

With the Mastodon dev team quite small, could it be worthwhile to request some help from the community in tracking the core issues down perhaps? There are lots of people on here who deal daily with high availability systems and should be able to figure this out. It requires the help of someone with lots of followers though to push some test content every now and then.

@_elena

@Renaud Chaput @Elena Rossini ⁂
in reply to Troed Sångberg

Elena Rossini ⁂
mastodon - Link to source
Elena Rossini ⁂
• •
@troed @renchap thank you for the response. I’m beginning to wonder if it’s a Ghost-specific issue because I never had that problem sharing links to my Wordpress sites… nor to my Ghost (Pro) blog… but every time I share on Mastodon a link to my self-hosted Ghost blog, the VPS gets immediately overwhelmed. Like within seconds. I installed Varnish cache but I’m still experiencing the problem
@Renaud Chaput @Troed Sångberg
in reply to Elena Rossini ⁂

Renaud Chaput
mastodon - Link to source
Renaud Chaput
• •
@troed When I make a query to your website, I see that the Cache-Control header has a `max-age=0`, which causes it not to be cached: `cache-control: public, max-age=0`
Did you update your Ghost config to include the caching options? ghost.org/docs/config/#caching
I would advise a maxAge of 10s to 60s at least for the frontend

Configuration - Adapt your publication to suit your needs

Find out how to configure your Ghost publication or override Ghost's default behaviour with robust config options, including mail, storage, scheduling and more!
Ghost - The Professional Publishing Platform
@Troed Sångberg
in reply to Elena Rossini ⁂

Steve Wart
mastodon - Link to source
Steve Wart
• •
does tooting with an image generally fix this problem?
in reply to Steve Wart

Elena Rossini ⁂
mastodon - Link to source
Elena Rossini ⁂
• •
@swart strangely no, so I’m beginning to wonder if it’s a self-hosted Ghost blog / Fediverse federation issue
@Steve Wart
in reply to Elena Rossini ⁂

patpro
gotosocial - Link to source
patpro
• •
if your VPS still collapses with proper caching in place, it might be a bandwidth issue.
in reply to patpro

Elena Rossini ⁂
mastodon - Link to source
Elena Rossini ⁂
• •
@patpro aw! What would you recommend doing?
@patpro
in reply to Elena Rossini ⁂

ティージェーグレェ
snac - Link to source
ティージェーグレェ
• •

The "traditional" way to mitigate the so-called "slashdot effect" (see: en.wikipedia.org/wiki/Slashdot…) other than optimizations (e.g. just serve up static content which reduces server load, or buy and use servers with faster CPUs, more cores, more RAM, etc.) is to use a CDN (Content Distribution Network, see: en.wikipedia.org/wiki/Content_…). There used to be a free/academic CDN called CoralCDN, which was pretty awesome; but I don't think anything similar has replaced it?

OFC, BitTorrent and PeerTube and other p2p paradigms obviate the utility of a CDN, but they tend to require a bit more effort to set up.

CC: @patpro@social.patpro.net

effect that occurs when a popular website links to a smaller website, causing a massive increase in traffic

Contributors to Wikimedia projects (Wikimedia Foundation, Inc.)
@patpro
This entry was edited (2 weeks ago)
in reply to ティージェーグレェ

Elena Rossini ⁂
mastodon - Link to source
Elena Rossini ⁂
• •

@teajaygrey thank you for the advice! @renchap suggested I should look into my Ghost caching settings: ghost.org/docs/config/#caching because they seem to be badly configured.

it's really bizarre what keeps happening whenever I publish something on here because my VPS plan offers 4TB of bandwidth and I use so little. Like, it's basically undetectable in my dashboard. I'll dig into the config files, make the suggested tweak and do some A/B testing


Configuration - Adapt your publication to suit your needs

Find out how to configure your Ghost publication or override Ghost's default behaviour with robust config options, including mail, storage, scheduling and more!
Ghost - The Professional Publishing Platform
@Renaud Chaput @ティージェーグレェ
in reply to Elena Rossini ⁂

Renaud Chaput
mastodon - Link to source
Renaud Chaput
• •

As I explained, you set up Varnish for caching, but your Ghost install is configured to not allow any caching. This is not a bandwidth problem, once you have proper caching configured all should be fine.

@teajaygrey

@ティージェーグレェ
in reply to Renaud Chaput

Elena Rossini ⁂
mastodon - Link to source
Elena Rossini ⁂
• •

@renchap thank you Renaud! 🙏 I just tried adding the caching code in config.production.json but got a JSON error message... will try figuring it out.

One last question: how did you see that the MaxAge was set to 0? (like what do I need to do to check it myself)?

@Renaud Chaput
in reply to Elena Rossini ⁂

Renaud Chaput
mastodon - Link to source
Renaud Chaput
• •

With curl, the best tool to troubleshoot those issues.

By running `curl -I https://news.elenarossini.com/`, it will display the HTTP headers sent from your server for your homepage. You will see that the `Cache-Control` header sets a max age of 0 (= do not cache), and that the `age` header (set by Varnish) is 0. If caching works, you should see a `age` value greater than 0.

in reply to Elena Rossini ⁂

patpro
gotosocial - Link to source
patpro
• •
first I would diagnose, there might still be another limit (memory / socket number...).
I'm not super comfortable giving advice about a virtual server I know nothing about, but diagnose a bandwidth limit is hopefully straight forward: you probably pay for a bandwidth allowance so you could compare what comes out of your server to that allowance. A local script logging Tx and Rx for your network interface every 10 seconds should do the trick.
Take a look at your log files too, you might discover an explicit limit that is not bandwidth related (too many open files, for example). Having data before / during and after an incident is precious.
Further actions will depend on the root cause, obviously. You could pay for more bandwidth and / or shrink your media files, or you could add some RAM to accommodate more simultaneous connections, etc.
in reply to patpro

Elena Rossini ⁂
mastodon - Link to source
Elena Rossini ⁂
• •

@patpro thank you! for additional context: mastodon.social/@_elena/114261…

Elena Rossini ⁂

2025-04-01 07:21:16

mastodon - Link to source

@teajaygrey thank you for the advice! @renchap suggested I should look into my Ghost caching settings: ghost.org/docs/config/#caching because they seem to be badly configured.

it's really bizarre what keeps happening whenever I publish something on here because my VPS plan offers 4TB of bandwidth and I use so little. Like, it's basically undetectable in my dashboard. I'll dig into the config files, make the suggested tweak and do some A/B testing



Configuration - Adapt your publication to suit your needs

Find out how to configure your Ghost publication or override Ghost's default behaviour with robust config options, including mail, storage, scheduling and more!
Ghost - The Professional Publishing Platform

@patpro
⇧