Challenge: Healthy substrate formation for the Fediverse

[aschrijver]

I bumped into a German Diaspora* discussion (found via this fedi thread) ) that led me to (re)discover a bunch of blog posts. At the time I first read them I knew only little about the Fediverse. There are valuable insights in the observations made by Dennis Schubert about ActivityPub. Dennis is an engineer currently at Mozilla and previously involved for 11 years with the Diaspora* project. He wrote:

• ActivityPub - one protocol to rule them all? (2018-02-01)
• ActivityPub - Final thoughts, one year later (2019-01-17)
• The 2010s and alternative Social Media: A decade full of work, hope, and disappointment (2020-01-01)

And independently Cory Slep of Go-Fed in An ActivityPub Philosophy draws similar conclusions, and mentions an interesting idea. The points raised in these articles align with many of the things that worry me regarding ActivityPub’s future and the ability for the Fediverse to reach its full potential.

Summary

• It is best to position ActivityPub et al as a framework for building social networking applications.
• ActivityPub is a transport protocol and advocating “Add AP support and join the Fediverse” is raising wrong expectations.
• Each and every implementation needs well-defined extensions for the domains it is in, to ensure broad interoperability.
• Fediverse evolves solely based on Ad-hoc Interoperability and lacks the substrate, people and processes, that are needed.
• Key challenge → Re-establish a healthy substrate, ensure substrate formation where Fediverse enters new areas.

I took the notes below mostly for myself, but you may find them interesting…

Social network protocol vs. framework

Some observations made in the first article (paraphrasing, and not all points mentioned):

• Theoretically AS/AP flexibility make it possible to represent any kind of interaction.
  • Practically this makes it very hard to create server implementations and UI’s that handle this.
  • Allowing basically arbitrary JSON is a huge danger for compatibility between applications.
• ActivityPub Client-to-Server (C2S) is an interesting way that could bring solutions to this flexibility.
  • In practice projects will likely implement their own customized payloads, and clients not be interchangeable at all.
• Security related mechanisms aren’t part of the specification (which was an intentional choice).
  • Authentication, Authorization, Verification, Encryption must be well-defined in a distributed social network.
• Fixed relationship model (following / followers, etc.) between actors is limiting.
• In general ambiguity introduced in the specifications leads to more complexity and edge cases.

Concluding Dennis’ opinion on an open standard specification in general is:

• Specification should be very strict covering all possible object types, properties, etc. leaving no room for interpretation.
• A “Living Federation Protocol” is needed where maintainers add extensions after discussion and further consideration.

In the follow-up “Final thoughts, one year later” Dennis first highlights:

• AP co-author Christine Webber’s description on the standardization process, which was intense, fraught and frustrating.
• Response by Friendica contributor Michael Vogel to Dennis’ first article, point for point explaining that “It is not all bad”.

Then continues stating:

• ActivityPub is not a full social networking protocol, but a transport protocol to transfer things (analogy: postal office).

And now importantly defining ActivityPub (paraphrasing again):

ActivityPub and related standards are a framework for building federated social network interactions, not an actual protocol.

Cory Slep also calls it a specification for “a transport protocol that alone is not sufficient to build an application”.

I often call ActivityStreams a “toolbox of social networking primitives” that can be built upon. But considering the whole of AS/AP as a “framework” may be more crucial in order to evolve the ecosystem built on top of these specifications!

Expectations and practice: Ad-hoc interoperability

Dennis Schuber then points to a flaw in the communication that subsequently occurred by advocates of AS/AP:

“[Advocates] wandered around the internet and told projects to implement ActivityPub to join the Fediverse, and by merely adding ActivityPub support, users would be able to communicate with each other, they claimed.”

And here Dennis comes the observation that ActivityPub in practice lacks “By-design Interoperability”, which matches my observation that since standardization any evolution of the specifications are in the form of Ad-hoc Interoperability (i.e. extending on app-by-app basis).

In fact all across the board, i.e. the Fediverse, wrong expectations are raised of widespread interoperability resulting from an implementation of ActivityPub support. This is impossible given the full range of social networking use cases. Vocabulary extensions can only be interoperable if everyone interested to adopt them agrees on the format / heuristics used.

These wrong expectations are a bit similar to how the Semantic Web was pitched, promising universal semantics that would turn the whole web into a machine-readable and navigable graph. A promise that IMHO led to its failure at the time of introduction (and hype).

Existing implementers of ActivityPub may be very happy about the possibilities on a technical level, but for newcomer developers and especially for the (would-be) fedizens it is vital that expectations are properly managed. Communicating differently about the Fediverse and a more appropriate positioning should be seriously considered. This aligns with Challenge: Fixing the Fediverse Technology Adoption Lifecycle.

At the end of the article a great explanation of Ad-hoc Interoperability is provided by Dennis:

“The issue arises when you have to read other project’s source codes and when you have to ask questions to be able to implement something. If that is the case, then the knowledge needed to implement this protocol reliably is no longer collected inside the specifications, but in other people’s bug trackers, source code snippets, and brains. Without access to that information, you cannot build a reliable implementation.”

This is the state of the Fediverse: 4 years of Ad-hoc Interoperability layered on top of the specifications!

I strongly urge to read the entire paragraph “But we can talk to each other to make it compatible”, but I will rephrase the inevitable outcome of our current approach to fedi’s evolution:

The levels of interoperability of a federated application with any other implementations will depend on Luck.

To give just example here… Lemmy’s federation release was lucky to interoperate with Mastodon given the current state of the Fediverse, and later on the federation broke again, when the state evolved organically.

Note: There’s also Noel de Martin whho describes some different forms of interoperability in the article Interoperable Serendipity.

Fediverse needs Substrate to survive evolve

According to Cory Slep “Anything more than just sending and receiving information needs to be built on top of ActivityPub as an extension” (confusing here is that the AP spec is interwoven with its own extension: for microblogging). He calls the domains targeted by an extension a “flavor” and hence will create a “Fediverse of Flavors”. He says:

"[It is] preferable that each domain of the Fediverse builds one or more independent communities to standardize their specific solutions through the use of ActivityStreams extensions. […]

Unfortunately, when the ActivityPub protocol was published there was neither guidance nor support on how to build these communities, how to develop more specialized domains, or how to organize more rigorous behaviors around the data being shared. And there still isn’t."

These communities and their collaboration and cross-pollination I call “the substrate” of the Fediverse that must exist. The ‘substrate’ of an ecosystem such as the Fediverse are the people and processes and the artifacts they create that enable interoperability. The more decentralized, diverse and heterogeneous the ecosystem, the more need exists for a strong and healthy substrate to uphold it.

Dennis Schuber mentions XMPP as a example of a specification where 'Extensibility and Interoperability are closely guarded:

• “they wrote a very strict base set of the absolute minimums required to build on XMPP” (IETF RFC 6120)
• For definition of payloads they introduced XEPs, XMPP Extensions Protocols. And XEP-0001 describes the process for doing so.
• Everyone can submit a proposal, and when following the rules, it is considered and when community finds it useful, adopted.
• All extensions are available at a central location, and are managed by the XMPP Standards Foundation.

On the Fediverse a central body that manages these specifications is a sensitive subject. There’s resistence to any centralization.

The Fediverse at this moment unfortunately does not have a healthy substrate. It is evey developer for themself, look into codebases and issue trackers, try to find other developers to ask a question. W3C SocialCG and SocialHub community, initially very active, have lost momentum and relevance. Though a Fediverse Enhancement Proposal (FEP) process was started, it hasn’t really gotten underway.

Cory Slep formulates two problems to overcome:

“The first problem is getting different ActivityPub communities to self-organize to solve their own domain problems, build their own specifications on top of ActivityPub, and hold each other accountable at sticking to those specifications rigorously. A second problem is discoverability of these specifications for fellow developers without having a centralized organization hosting a registry or dictating a process. The cost of extending ActivityPub should always remain as close to zero as possible for numerous philosophical and moral reasons that I don’t want to get into here.”

Without a substrate the only way for a project to move forward is by Ad-hoc Interoperability. And once an app is successful there are less incentives to dedicate to substrate formation. There are already enough things to take into account to keep a FOSS project going.

Solutions for substrate formation

And in his article Cory poses an idea that really delights me, as it corresponds to directions I was also thinking in:

"Why not build on top of ActivityPub to solve both of these problems?

First, we as a community create an ActivityStreams extension and define behaviors to document the specifications built on top of ActivityPub. We now have a way to store ActivityPub extension specifications in the Fediverse itself. Next, building an application server that understands this extension and can federate it over ActivityPub means anyone can simply host an instance, allow users to register, build its community, and be free to draft new specifications on top of ActivityPub. Since this data is then federated by ActivityPub itself, it is reachable by both technical and non-technical users. There is neither a central authority nor a central registry as each community is responsible for its own ActivityPub specifications. This is the democratization of ActivityPub across all domains. People cannot have their ActivityPub extension specification censored. As long as the core ActivityPub protocol as currently written is followed, federation compatibility naturally leads to new domains and apps implementing them."

I proposed something similar a while ago in: Improvement to FEDERATION.md convention: Murmurations.

Note that these solutions are only a first start. It allows people to get informed of specifications and documentation, but in future more more machine-readable mechanisms are also needed, such as e.g. Capability Discovery and Compliance Profiles.

This does nothing to address the social aspects of substrate formation. Objectives are to:

• Lower the barrier to participation as much as possible.
• Automate as many of the chores as possible.
• Highlight the incentives and win-wins of participation.

Related to these social aspects, in his last article Dennis Schuber highlights a point I highly agree with: Overall those developing the Fediverse are way too tech-focused! Note that the Social Coding Movement was started to address this:

Social Coding

Create solutions that people love. Social Coding is participatory free software development that is inclusive to all people and focused on addressing real human needs. Ideation, design and development is a collective and social effort.

As for positioning the Fediverse, I am personally promoting the idea of conceptually creating a more human ‘abstraction layer’ on top: the Peopleverse. Where the social interactions that are modeled are key, and not all the technical jargon that is now in our ‘elevator pitch’.

(Image credit: creativity103.com, “angled metal tracks on an electronic circuit board”, CC-BY)

[aschrijver]

@aschrijver

ActivityPub and related standards are a framework for building federated social network interactions, not an actual protocol.

Exactly on this point a new fedizen (Google SRE employee) is coming to the same conclusions:

Hrefna (DHC) (@hrefna@hachyderm.io)

Their timeline around this point are full of similar analyses regarding the protocol and its flaws and shortcomings.

[Mediaformat]

I’ve heard this before, that ActivityStreams/ActivityPub is underspec’d to be a protocol proper, but I’m curious in what way this also can’t be said about RSS which is considered a protocol?

[aschrijver]

I don’t know enough about RSS to compare. There are a bunch of Google employees exploring the fedi, and @hrefna recently toot about findings that relate to the AS/AP mixup of different concerns:

Hrefna (DHC) (@hrefna@hachyderm.io)

[IgnisIncendio]

@aschrijver

Related to these social aspects, in his last article Dennis Schuber highlights a point I highly agree with: Overall those developing the Fediverse are way too tech-focused! Note that the Social Coding Movement was started to address this:

Social Coding

Create solutions that people love. Social Coding is participatory free software development that is inclusive to all people and focused on addressing real human needs. Ideation, design and development is a collective and social effort.

As someone with autism* I find it hard to understand such social interactions, ironic considering I am in a forum named Social Coding . I am actually quite asocial and a lot of effort and thinking goes into my social communications; I am one of the people who suck, half-joking. I would like to point out that there is a technical solution to this: bridging.

With bridging, no cooperation is needed as long as protocols are open. I’ve recently also been thinking about the effectiveness of social solutions. I feel that they are ineffective. I have written a few examples below, but I’m sure there are counter-examples I have not thought of.

• Drew DeVault calls distro packaging a social solution but it’s unable to keep up, which is causing more apps to switch to Docker/FlatPak/web apps – sandboxed instead of human-checked.
• Usenet having conventions that was overwhelmed by the Eternal September.

*Yes, some can xD but I find it very hard. It’s a spectrum, yo!

[IgnisIncendio]

Originally posted in #SocialCoding:matrix.org

Ah, I’m starting to understand now. For example safety in civil engineering is a systemised social process — an engineer needs to check, sign off etc.

XMPP and Matrix are examples of systemised social processes, although they differ in that one is based on XEPs and the other on layering new functionality. For example FluffyChat has Stories functionality, and that can be standardised instead of ad-hoc compatibility.

If we bridge XMPP and Matrix, it is a technical solution + social solutions.

However currently the Fediverse is an unsystemised social process, which should be systemised. Tyranny of structurelessness here too.

Or, quoting CGP Grey:

Additional info

Right now I’m leaning towards “social solutions + technical solutions, but if you can only choose one, choose social; I would sooner trust a spacecraft audited in MISRA C than one unaudited and in Rust”.

[aschrijver]

In follow-up to Ignis’ comment later in the discussion Anthony Wang remarks that FEP’s already serve to document existing AP protocol mechanisms:

That’s to some degree what FEPs already do, for instance, the NodeInfo FEP or the group federation FEP.

And I follow up with:

Yes, indeed. Note that both the actual open standards and FEP’s etc. can be seen as part of the “technology substrate” (or “ecoystem substrate”, either will do) but that it also includes the people and processes to evolve those. So @IgnisIncendio, your idea of documenting FEP’s of what’s already out there is good, but who has incentive to commit to that huge chore, and who will subsequently maintain and update it? That’s part of healthy substrate.

@IgnisIncendio reply:

Ahh, I see. Makes sense. “What motivates the people?” Matrix does so by being a company paid with Element. XMPP…? Not too sure. Donations/patronage if people recognise the good they put into the world? Thanks for the response :)Hmm. Working groups… reminds me of such in social.coop. (I don’t think this thought goes anywhere tho)

My reply:

Yes, there are many ways such substrate can be organized. The W3C, for instance, does so in a very formal manner, and - I think until recently - receiving money from MIT to be able to afford having people dedicated to running these processes, plus likely donations from various parties. On the Fediverse people in general have aversion to something so centralized and authoritative, whether it is justified or not. So a different form of organization may work better. This is something I have been thinking about, as I’ve seen the unwillingness of many folks to contribute to SocialHub to evolve the substrate. What imho is an important factor is to take into account dynamics and culture.

Where you mention Working groups they may as well be decentralized groups of people working in a particular domain. This way they all have incentive to see the domain progress, as it directly benefits their project. What remains is a common denominator - the core mechanics of the protocol - that SocialHub could dedicate to. I proposed such scope as well.

POLL: SocialHub Scope and Purpose? - Community - SocialHub

The problem imho with any (much) bigger scope and audience is that you need dedicated people doing the chores of keeping the community together, and that the majority of things discussed in the community are only relevant to a subset of its members. Hence they are less involved, less likely to be active participants.

In the Solidground project there is another idea for helping evolve the substrate. Solidground has this social experience designer called Floorplanner. With it people will model their domain (it is based on domain driven design), and part of that domain model will become an ActivityPub vocabulary extension (for those domain concepts that are federated). With Floorplanner you design the Blueprint of a social experience. This blueprint consists of Documentation, Specifications and Configuration. All these 3 things become bundled within the social experience, that is deployed as a Service Module. By doing all this there’s more assurances that a) docs / specs / code are in sync and b) are available at the source where functionality is offered, should others want to integrate with it.

[aschrijver]

Manton Reese, developer of micro.blog, wrote a blog post Next step for ActivityPub (archive link) that comes to the same conclusions as Dennis Schubert did years ago (and which gain more traction in dev community nowadays):

To frame this blog post I’ll put forward this question, at the heart of interoperability on the fediverse:

Is it possible to implement a social web platform by reading the suite of ActivityPub specs, and have that new platform be compatible with Mastodon?

I would argue no, it is not possible. A new project would only be compatible with Mastodon by accident because there are some things that are not spelled out in precise detail in the specifications, and no JSON examples that exactly match what any server sends or expects. The specs have all the pieces, but not how to fit those pieces together.

To relate to this forum topic: Only a healthy technology substrate is able to fix this problem. Though this substrate may have tool support to make that easier.

[aschrijver]

Cross-referencing to my effort of improving what I dubbed the “Grassroots Fediverse” entailing just a tiny bit more coordination between all the fragmented and independent initiatives that are popping up (and many of them dying again) all of the time:

Ideating organization structure for the Grassroots Fediverse (wiki)

I sketched the basic idea in a diagram, included here:

Until now after plenty efforts to bring attention to this, if there’s one conclusion, it is this:

There’s hardly ANY interest by people to collaborate beyond their own initiative.

This is the major characteristic that defines grassroots movement, i.e. “herding cats”.

• “Only my initiative interests me.”
• “I can do this better than you.”
• “I don’t want to coordinate, but be independent.”

(Hence I will not put any further energy in this proposal. It is a waste of time)

[aschrijver]

State of the Fediverse substrate 2024

In 2023 there was a big uptick in people investigating how to improve the Fediverse DX. Despite all efforts there’s much to be done, and progression on the “technology substrate” was but small.

There’s a lot to be proud of on what the commons has achieved collectively. I am also very happy on how much the FEP process picked up steam, and the overall willingness that exists to improve it further. OTOH I am deeply worried about the corporate entries speeding up, and fedi a sitting duck to a takeover. As I mentioned often before there’s good things to grassroots slow and organic evolution. First it is vital I think for healthy culture to be fostered collectively. That is a naturally slow process. Corporations OTOH have “disruption” among their strategies. Also the emergence from chaos in many fragmented initiatives brings a certain resilience to resists that disruption.

AS/AP was meant to be an interoperability protocol. One to unite fragmented efforts. It’s evolution needs a “technology substrate” of people and processes to evolve it. While new apps may be evolving the fedi, at the same time - by lack of substrate and making ad-hoc choices - they introduced protocol decay (tech debt at substrate level) and by doing so there’s an element of devolution to the protocol itself and its interoperability objectives. I coined the term WDD, Whack-a-mole driven development, of how you need to maintain app-by-app integrations against moving targets.

The only thing countering full devolution into ad-hoc interop is an - also grassroots and mostly implicit - design-by-consensus process that forms our current “technology substrate”. It consists of numerous discussions in countless channels among different people, and no decision-makers other than devs picking nuggets to add to their codebase if they happened to listen in on some of these discussions.

Even this process in itself is not bad. See aforementioned qualities of grassroots movements. But they don’t stand on their own. They stand the scrutiny of the ‘outside world’. Competing protocols that can organize their substrate with the ability-to-act better assured and working on lessons-learned with greenfield material they can introduce in well-controlled stages (e.g. ATProto) and corporations that can throw a huge amount of paid workers and resources at something and dictate “this is how it is done” and matching their corporate interests first and foremost.

The biggest pain point in ActivityPub is its flexibility and the lack of guidance to deal with that. ActivityPub really is not a protocol, instead it is the outline of a protocol framework. The biggest lodestone to progress imho is saying that AP is Linked Data and LD is the extensibility mechanism, but then subsequently totally failing to provide sufficient guidance in how to create quality extensions based on that. The LD spec experts basically left the field as soon as AP was deemed a Linked Data standard, with a handwavy “see the LD specs”, and left fedi devs to figure out a LD ecosystem which after many, many years is still a big mess and immature.

Now, all of this flexibility still isn’t the heart of the problem. It is widely acknowledged, discussed, it is being analysed, and people with the expertise can distill ways to properly deal with it. The real problem is “Who does the work?”. It is not just providing the guidance, but also advocating its adoption, and in total it amounts to a ton of work. Not easily organized by volunteers in our chaotic environment. And currently there’s no real funding either for this kind of work.

In latest funding rounds by NLnet and STF three different AP test suite projects received funding independent of each other. It is great, much needed, but they are already faced with a “what are we even going to test?” without the fundamentals of the protocol being crystal-clear. We don’t know the ‘domain model’ of the protocol, the layers of the protocol stack, and best-practices for extensibility, or even use cases / applicability of AS/AP. Different people have entirely different views of what the protocol is for.

All these things above make me say what I said yesterday. And saying them for years now alongside calls to collectively address this. But here I notice people rather launch their own initiative, do it their own way, in true splinterverse fashion (yes, exceptions exist ) and we’ll see where we end up.