Skip to main content


This seems very important and worth ongoing study:

“Once again, results suggest a rise in diversity as the 10 biggest server contribution to the Fediverse is reduced by more than 10%. So, even if the biggest servers are accumulating more users, it seems that the Fediverse is becoming more decentralized.”

@fediversereport @spreadmastodon @fediversenews

https://socialhub.activitypub.rocks/t/analysis-of-fediverse-diversity-in-terms-of-decentralization/3252

reshared this

Interesting! Although there are some quirks in the data, with joindiaspora and diasp.org (neither of which are Mastodon) in last March's accounts and not the current list, and with mastodon.cloud and gc2.jp going from over 10% of MAU to not appearing at all in the latest statistics.

Also, it'd be nice to know something about what comprises all the "others", how many accounts do those instances have, how many of them are there?

Otherwise, it'll be interesting to track this going forward because mastodon.social right now is growing faster than it did between March and 17 May ... the picture could very well look different when comparing May to August.

>Also, it’d be nice to know something about what comprises all the “others”, how many accounts do those instances have, how many of them are there?

Others means all the rest! Which means 21089 in May (as shown in the first table).

>Otherwise, it’ll be interesting to track this going forward because mastodon.social right now is growing faster than it did between March and 17 May … the picture could very well look different when comparing May to August.

Totally true! I would like to take monthly pictures (with the help of @spla, which is the author of the API query script).

Thanks for the reply!

Any chance others can get their hands on the data set?

also, another question … any insights on your your data set and its creation would differ from any of the others out there like fedidb or instances.social?

Hey! That is interesting... I didn't thought in using fedidb (the other one I didn't know). The truth is that @spla took the data by itself and, as I had the chance to look at it, performed the analysis.

It will be interesting to do the analysis with the fedidb dataset. For what I see right now, it seems that it differs from the dataset used by me. I can see an increase in servers in Oct 22 that results in a decrease in Users by server, and then it keeps more or less stable.

I would like to apply the shannon and simpson indexes and the top10 server distribution, as they gives a broather view of diversity.

Plot showing Fedivers server amount through time.
Plot showing Fedivers User per server ratio through time.
I am playing with fedidb API and I think I could get all the data I need (first time using APIs myself!).
Yea the API works well ... I've used it myself. Last time I used it though I think there was an issue in the data not having many of the small (1-10 user) instances. But from the dashboard that seems to have been fixed now.
When I find some time, I will try to recover this global data. I find particularly interesting the ecology measures of diversity to be applied to user distribution and software distribution.
What do you mean by "global data" ... what are you intending to recover?
The same as spla did, software, user and mau data from all servers included in fedidb.

Ahh ... right, collect the data yourself.

It does strike me though that it's the sort of task that we'd be better off doing more collectively. We could pool the algorithms to get the best one and collect multiple datasets from multiple origins to maximise coverage which can then be merged.

Also, just in case it's useful, here's my quickly hacked together python code for getting all the data from the fedidb API:

# +
from collections import deque
import requests as req

base_url = 'https://api.fedidb.org/v1/'
servers_url = f'{base_url}servers'
# -
# +
server_data = deque()
params = {'limit':40}
n = 0

while True:
	if n%5 == 0:
		print('Loop', n, 'servers', n*40)

	r = req.get(servers_url, params=params)

	if r.status_code == 200:
		d = r.json()
		server_data.extend(d['data'])

		next_cursor = d['meta']['next_cursor']

		if next_cursor is None:
			print('Cursor is None ... FINISHED')
			break
		params['cursor'] = next_cursor

	else:
		print(f'request broke and returned {r.status_code}')
		break

	n += 1
# -
# +
len(server_data)
server_data[0]
# -

Thanks! I'll take a look on that!

>We could pool the algorithms to get the best one and collect multiple datasets from multiple origins to maximise coverage which can then be merged.

That would be cool for sure, although I'm not sure I can add much in the technical part (I am a biologist with some data analysis skills).

(I am a biologist with some data analysis skills)


... me too!

I recently did some analysis of my own using data from instances.social, but didn't have any historical data to compare to. I'm intending to make a comparison after a month or too.

You can see my analysis here: https://hachyderm.io/@maegul/110331433071884694


Graphs of the sizes of fediverse instances, how common they are, and where the most people are! 🧵

Data pulled from https://instances.social/ (by @TheKinrar) and excludes pawoo and baraag as they're heavily blocked for good reasons (it seems)

Breaking down instances by the number of users into bins (that are quasi human friendly logarithmic), we see that the majority (55%) have 2-50 users, ~33% have 1 user, and almost all instances have less than 5,000 users.

@fediversenews

1/

Histogram of fediverse instances by account size ... see toot for description

The API must have changed, because after 3960 servers the code breaks. I have been trying by myself with R and more or less happens the same…

unfortunate. They make no promises about stability. Though from memory, I think the same thing happened to me when I used, and I figured their data was incomplete.

I asked the developer about it (they make pixelfed) but gotten no response.

@marcelcosta I agree, applying those ecology measures of diversity is very valuable -- I really appreciated that in your post!

Does the FediDB API let you get at historical data? Or, is the historical dataset available?

@maegul @tchambers @fediversereport @spreadmastodon @spla @fediversenews

Not that I know of ... if so it would be a hidden/undocumented feature.

In the end, given that these datasets aren't terribly large, especially if we have a temporal resolution of only 1 month or week, I think it's getting to the point that it'd be nice if they were just stored somewhere easily accessible to all.

@maegul I get the dataset from all peers of my server and then from all peers of my server's peers. After collecting all those peers, my code get the nodeinfo URL of all of them and ask it for users and MAU data, so only *alive* servers data is saved to the dataset.
Here is the code https://git.mastodont.cat/spla/stats
The bot @fediverse is publishing global registered users and MAU.

@marcelcosta @tchambers @fediversereport @jdp23 @fediversenews @spreadmastodon

Awesome! I've already been following that bot. Thanks!!

@maegul you are welcome! by the way, you can ask any server or soft data to it, check its profile to see how you can do that.

@fediverse @marcelcosta @tchambers @fediversereport @jdp23 @fediversenews @spreadmastodon

Wow, I found a huge mistake in my analysis!

The first time point it's not from March but from September! I am a bit embarrassed, but this is what happens when you rush because of time (and also, chaos in date format).

In fact, this makes more sense and is also interesting because it seems that some months after big October wave, it seems that Fediverse is more decentralized.

I've got the impression that it was somewhat necessary that mastodon be more decentralisation in order to take on all the new users as the larger instances, or some of them, struggled. mastodon.social, for example, didn't really gain more users from Jan to March, IIRC.

Sure, these waves have overloaded even not-that-big instances, causing them to close registration for a time. From September, the number of servers has increased by 3.

If we had a historical from the server picture, would be really interesting to trace the movements and evolution of the Fediverse…

That is interesting! There are snapshots from prior to Octobers wave and also from October, November, December, etc… So we can monitor what happened with that wave, what was the dynamics of users.

Yes! Even though the data's noisy and incomplete there are clear differences in dynamics in different time periods. For example (I think, if I did the math right)

9/20/2022 to 12/7/2022: mastodon.social + 71K
mastdn.social +80K

12/7/2022 to 3/25/2023:
mastodon.social +115K
mstdn.social +41K

3/25/2023 - 5/25/2023:
mastodon.social +91K
mastn.social +3.6K.

@marcelcosta @maegul @tchambers @fediversereport @spreadmastodon @fediverse @spla @fediversenews

Yea ... just squinting at the graphs, I think these errors in the graphs mask the growth of mastodon.social. I would bet that when corrected, mastodon.social will have grown in it's share from March to May.

@Jon Did you seriously expect a #Fediverse analysis to be only about #Mastodon?

#MastodonIsNotTheFediverse. And misskey.io is not Mastodon either; it's the biggest #MissKey instance.

@Tim Chambers It's a pity that only a bunch of instances were analysed that were important in mid-March. The graph would look different if it had taken the recent #CalcKey growth into consideration.

@jupiter_rowland No, I didn't expect it to only to be about Mastodon -- in fact the article was very clear that "the active account data refers only to Mastodon servers, as other software don’t necessarily give this information". I was just highlighting that two large instances that disappeared from the stats and neither of them are Mastodon, so that implies something janky going on with the stats.
@tchambers

@Jon @Tim Chambers Well, as for Joindiaspora, it's currently shutting down by and by as announced. It's only kept online for users to export their account data, but posting is no longer possible.

In general, maybe Diaspora* has been taken out of the equation because it's only connected to a few projects on the fringes of the Fediverse. Diaspora* and Mastodon users can only interact in comment threads on e.g. Friendica or Hubzilla, if at all. Nonetheless, if it has been in the March analysis, it should have been in that from May as well. By the way, diasp.org is very much alive.

@jupiter_rowland @jdp23

I'd say: it's a start...and I could see from an analysis standpoint how the researcher started with what was doable directly via the #mastodon API. But hopefully to add others as they can.

Adding the author into this thread: @marcelcosta

And by definition, I'd think the decentralized story is likely only *better* when you assume the other non-Mastodon Fediverse data... the-federation.info now counts Calckey as 3,500 users, I think FediDB counts them as 8.700 users.

@Tim Chambers @marcelcosta @Jon If either are accurate. All I know is that CalcKey is growing like crazy because it gets a whole lot of exposure and publicity in the Fediverse, and because it's currently the most feature-rich ActivityPub-based project while still having a decent UI.

In this regard, I think the refusal to include Akkoma and CalcKey because Pleroma and MissKey are already there isn't quite smart. Pleroma barely matters because Akkoma outshines it, and MissKey only matters in Japan while CalcKey is booming over here.

Hey! That was the point of the analysis in part, to generate debate and tools to monitor.

In theory, absolute accounts of servers include data from many softweres. It's the MAU value that only includes Mastodon servers. I think that both measures show the same trend, so.

And yes, API query must be improved. Some diaspora servers are excluded because of lag in answering. This should be addressed (although the biggest instance is alive but will close soon and doesn't accept new posts).

If there is interest on that, we can plot software distribution across servers and users.

@marcelcosta @jupiter_rowland @jdp23

Thank you Marcel! And appreciate your joining into this discussion on your work!

You're welcome! This was my intention. Me and others are really interested in measuring decentralization and network quality over quantity.

@marcelcosta @jupiter_rowland agreed, it's useful data despite the quirks-- and there's clearly interest!

There are several other measures of software and instance diversity that seem interesting to me: Mastodon's share of the overall Fediverse, mastodon.social's share of Mastodon, the top 10 instances' share of Mastodon, percentage of single-user instances and so on.

Maybe we could group all this questions in the socialhub threat (or here, if people don't have accounts there, although discourse is great to keep track of discussions).

I did a previous analysis (with serveral limitations) with March data, although it is in catalan language.

https://agora.fedi.cat/t/analisi-dels-servidors-del-fedivers/617

@marcelcosta it's fine with me if people want to cut-and-paste the questions to the socialhub thread but I like my discussions about the #fediverse to take place in the fediverse! I agree that discourse is currently much better to keep track of discussions but there's no inherent reason the fediverse should be bad for that and "eating our own dogfood" creates pressure to improve things here.

@jupiter_rowland @tchambers

Good point. I will have to bookmark the threat to revise when I want to repeat the analysis. 😄

@jupiter_rowland Is there actually a refusal to count those? Or is it just a combination of measurement artifacts (some don't show users) and that their numbers are too small to show up at this point? Calckey's growth for example is from a tiny starting point.

@tchambers @marcelcosta

>Is there actually a refusal to count those?

No, as far as I understand, if the API from the software returns users, there are counted. Akkoma is included for sure (and to note, I'm writing from a server with it :) ), not sure with calckey, but we can check it.

@Jupiter Rowland

In general I find it noteworthy that pretty much no one is talking about the phenomenal growth Akkoma has had since its launch, which was just over 1 year ago. Comparatively it is really impressive.

It is twice the size a certain other service that "everyone" talks about (and the same amount of MAU's, which admittedly speaks for the even newer service), and which I also wish well, but given the talk about it I would've expected it to be explosive...not half the amount of users of Akkoma.

I just find it noteworthy that there isn't a beep regarding Akkoma (unless it comes from me, and people are bored of me raving about it).

I wonder what would've happened if Akkoma had gotten the same exposure though, given that it has a similar feature set.

#ImNotBitter 😆

@Tim Chambers @Jon @marcelcosta

reshared this

@fediversenews

I appreciate the need to make visible many software. This is part of the decentralization! I can do a second round of analysis looking at these. However, in this first part I focused in the user distribution between servers. I did a first analyisis including the software information (I have shared it, although is not in english), but will be interesting to see the dynamics, too!

I have to say that I did this analysis in my free time, so I am sure that many things can be improved!

Fediverse News reshared this.

@marcelcosta @Fediverse News Oh yeah, sorry, I just read the comment from @Jupiter Rowland and felt I needed to air this thought I've had for a while, I forgot his reply was in relation to this visualisation.

I find your analysis really interesting. I didn't mean to give critique to that. I was just doing a fly-by comment of a general observation.

Fediverse News reshared this.

@jdp23 @tchambers No problem! Indeed, it will be interesting to see the adoption of all software across time. I have the feeling that I won't like the result, but I thought that the Fediverse was getting more centralized and I was surprised by the results... So, ho knows until we check it?

Fediverse News reshared this.

@mathias @jupiter_rowland @jdp23 @marcelcosta

Please keep raving about Akkoma, Mathias, your comparative remarks are always interesting.

Regarding some of the other server types that are currently popular, the boundary between healthy enthusiasm and oppressive marketing hype is fluid, and it's good to see a degree of restraint and sobriety in the way in which servers such as Akkoma are discussed. :-)

Fediverse News reshared this.

@the roamer

Hehe, thanks, I usually take great care to not offend people (and am mostly successful in that…I have irony and sarcasm in my veins so it is at times not without challenges…), and I try to stick to verifiable facts and also give credit where credit is due, but I keep getting blocked by people here and there which I apparently have rubbed up the wrong way, so I’ve learnt I’m not for everyone (which is fine, more of an observation). 🤣

@Jupiter Rowland @Jon @marcelcosta @Tim Chambers

Fediverse News reshared this.

I’d say: it’s a start…and I could see from an analysis standpoint how the researcher started with what was doable directly via the #mastodon API. But hopefully to add others as they can.


Just a comment. @spla queried APIs from servers with many software installed, not just mastodon.

Love to see it!

A diverse and decentralized fediverse is a healthy fediverse.

Increased decentralization is great news; however, unless we all agree together to #FediblockMeta , the fediverse is about to become massively centralized, and mostly the private property of Mark Zuckerberg

@ophiocephalic
Agreed: growing decentralization is only a good thing for the Fediverse.

I do think we need to prepare and think through strategy on Meta for sure.

I do not thing reflexive fediblocking them should be our first tactic.

It almost always should be a last tactic only done at the most extreme situations of harassment and out-of-control content non-moderation.

More thoughts later.

Thanks for your response. Yes, this is an issue that requires much more conversation and thinking ahead.

I would respectfully suggest that everyone disinclined to block Meta pay attention to the intensity of determination of others of us to prevent our expressions from being absorbed by them. We are defederating. And to avoid our toots being siphoned up into his machine for surveillance and AI ingestion, we will need to defederate with every other instance that doesn't. The community as it exists today will shatter, and Zuck will have been allowed to destroy the fediverse. The only way to avoid this grim scenario is to treat Meta as another Gab, and unify in our rejection.

Meta is a bigger threat to the federation than any fascist instance; the fediverse has never faced a more extreme situation.

reshared this

Fediverse News reshared this.

@jupiter_rowland Agree that whitelisting a finite list of servers is a mistaken idea for all the reasons Jupiter lists..

But I’d restate that fediblocking for any reason other than clearly non-moderated instances abusing others is a misuse of fediblocking I believe…

@jupiter_rowland

and if an instance is fediblocking they should disclose the list of blocked instances on their about page.

it does not seem their is a direct way to see which instances have been block by yours.

https://www.reddit.com/r/Mastodon/comments/yzahsd/is_there_a_way_to_see_which_instanced_have/

Fediverse News reshared this.

@mathias @jupiter_rowland
Appreciate the attention to detail but wonder if this discussion got a bit lost in the weeds. No one wants a whitelist-only or Mastodon-only fediverse. What we are advocating for now is for us all to respond to Meta as we do to fascist/toxic instances. The technical details of implementing a schism are far less important that the big question - can we avoid it completely, and don't we want to?

Fediverse News reshared this.

@ophiocephalic @mathias @jupiter_rowland
I'd say keep powder dry on mass defederation of Meta's Barcellona till we see what exactly it is.

It could be mass horribleness, or it could be a great opportunity to stay federated with it, engage and migrate non-technical users out of it to better Fediverse platforms.

Wasn't it like less than a month ago that instagram was blocking a 'join pixelfed' hashtag?

Who's to say they won't block individual servers that try to start a migration, or refuse to deliver those posts?

They will not play by established norms, we know this based on 2 decades of experience with this monopolistic and manipulative corporation. There is no scenario in which they will accept losses.

@ophiocephalic @mathias @jupiter_rowland @spreadmastodon

Fediverse News reshared this.

Fediverse News reshared this.

@mathias @jupiter_rowland @ophiocephalic https://mstdn.patatas.ca/@smallpatatas/110435995213331020


The thing that makes the Fediverse amazing is not that it is perfect, but that passionate people are committed to continually improving it.

#Facebook/#Meta/#Instagram is ultimately committed to one thing only, and that is making more money.

They will not do something if they think it does not give them an advantage or if they think it will cause them to lose users.

There is no upside to federating with their servers.

#FediBloc


@mathias @jupiter_rowland @ophiocephalic @smallpatatas

I'd like to see one-way sharing. We let Insta share with us, but we absolutely don't share data that isn't fully anonymized back to them. We def. don't share IP like pictures without opt-in and we make sure that the disclaimer is explicit: If you opt-in, Insta (or whoever) may screw you.

This entry was edited (1 day ago)

@ophiocephalic @mathias @jupiter_rowland @J12t

I'd say consider and pre-plan for everything: but don't assume anything till we see what we really have here.

And we wouldn't need their permission to encourage users and to offer links to get folks to migrate over.

Even if they don't support migrating over the social graph as they can on Fediverse platforms offering links to and simple migration tools will definitely be a possibility...

Agreed that we don't know what if anything actually #Meta's going to do with #P92 (@J12t makes some great points) but it will be #SurveillanceCapitalism because that's who they are. And if the biggest instance in #Fediverse is a surveillance capitalism site, then the fediverse will likely (and justifiably) be seen as a surveillance capitalism network.

@ophiocephalic @mathias @jupiter_rowland @spreadmastodon

@ophiocephalic #meta won’t care whether they are #fediblock d or not. They do this for market positioning reasons vis-a-vis wanting to appear as the good guys wrt Twitter, not to actually interoperate with anybody. More detailed thoughts here: https://reb00ted.org/tech/20230522-what-the-meta-activitypub-not-so-fast/

All in favor of coming up with a coordinated response, when is your call? :-)

fedi won the mastodon war
I'd argue that it's a report on the diversity of Mastodon more than the diversity of the Fediverse...

May I ask why? I decided to analyze the MAU from Mastodon’s servers alone because I was suspicious that is not measured the same way across the softwares, so they might not be comparable.

However, the absolute account analysis (which reaches similar conclusions) include servers regardless of their software.