Challenge: Healthy substrate formation for the Fediverse


I bumped into a German Diaspora* discussion (found via this fedi thread) ) that led me to (re)discover a bunch of blog posts. At the time I first read them I knew only little about the Fediverse. There are valuable insights in the observations made by Dennis Schubert about ActivityPub. Dennis is an engineer currently at Mozilla and previously involved for 11 years with the Diaspora* project. He wrote:

And independently Cory Slep of Go-Fed in An ActivityPub Philosophy draws similar conclusions, and mentions an interesting idea. The points raised in these articles align with many of the things that worry me regarding ActivityPub’s future and the ability for the Fediverse to reach its full potential.

Summary

  • It is best to position ActivityPub et al as a framework for building social networking applications.
  • ActivityPub is a transport protocol and advocating “Add AP support and join the Fediverse” is raising wrong expectations.
  • Each and every implementation needs well-defined extensions for the domains it is in, to ensure broad interoperability.
  • Fediverse evolves solely based on Ad-hoc Interoperability and lacks the substrate, people and processes, that are needed.
  • Key challenge → Re-establish a healthy substrate, ensure substrate formation where Fediverse enters new areas.

I took the notes below mostly for myself, but you may find them interesting…

Social network protocol vs. framework

Some observations made in the first article (paraphrasing, and not all points mentioned):

  • Theoretically AS/AP flexibility make it possible to represent any kind of interaction.
    • Practically this makes it very hard to create server implementations and UI’s that handle this.
    • Allowing basically arbitrary JSON is a huge danger for compatibility between applications.
  • ActivityPub Client-to-Server (C2S) is an interesting way that could bring solutions to this flexibility.
    • In practice projects will likely implement their own customized payloads, and clients not be interchangeable at all.
  • Security related mechanisms aren’t part of the specification (which was an intentional choice).
    • Authentication, Authorization, Verification, Encryption must be well-defined in a distributed social network.
  • Fixed relationship model (following / followers, etc.) between actors is limiting.
  • In general ambiguity introduced in the specifications leads to more complexity and edge cases.

Concluding Dennis’ opinion on an open standard specification in general is:

  • Specification should be very strict covering all possible object types, properties, etc. leaving no room for interpretation.
  • A “Living Federation Protocol” is needed where maintainers add extensions after discussion and further consideration.

In the follow-up “Final thoughts, one year later” Dennis first highlights:

Then continues stating:

  • ActivityPub is not a full social networking protocol, but a transport protocol to transfer things (analogy: postal office).

And now importantly defining ActivityPub (paraphrasing again):

ActivityPub and related standards are a framework for building federated social network interactions, not an actual protocol.

Cory Slep also calls it a specification for “a transport protocol that alone is not sufficient to build an application”.

I often call ActivityStreams a “toolbox of social networking primitives” that can be built upon. But considering the whole of AS/AP as a “framework” may be more crucial in order to evolve the ecosystem built on top of these specifications!

Expectations and practice: Ad-hoc interoperability

Dennis Schuber then points to a flaw in the communication that subsequently occurred by advocates of AS/AP:

“[Advocates] wandered around the internet and told projects to implement ActivityPub to join the Fediverse, and by merely adding ActivityPub support, users would be able to communicate with each other, they claimed.”

And here Dennis comes the observation that ActivityPub in practice lacks “By-design Interoperability”, which matches my observation that since standardization any evolution of the specifications are in the form of Ad-hoc Interoperability (i.e. extending on app-by-app basis).

In fact all across the board, i.e. the Fediverse, wrong expectations are raised of widespread interoperability resulting from an implementation of ActivityPub support. This is impossible given the full range of social networking use cases. Vocabulary extensions can only be interoperable if everyone interested to adopt them agrees on the format / heuristics used.

These wrong expectations are a bit similar to how the Semantic Web was pitched, promising universal semantics that would turn the whole web into a machine-readable and navigable graph. A promise that IMHO led to its failure at the time of introduction (and hype).

Existing implementers of ActivityPub may be very happy about the possibilities on a technical level, but for newcomer developers and especially for the (would-be) fedizens it is vital that expectations are properly managed. Communicating differently about the Fediverse and a more appropriate positioning should be seriously considered. This aligns with Challenge: Fixing the Fediverse Technology Adoption Lifecycle.

At the end of the article a great explanation of Ad-hoc Interoperability is provided by Dennis:

“The issue arises when you have to read other project’s source codes and when you have to ask questions to be able to implement something. If that is the case, then the knowledge needed to implement this protocol reliably is no longer collected inside the specifications, but in other people’s bug trackers, source code snippets, and brains. Without access to that information, you cannot build a reliable implementation.”

This is the state of the Fediverse: 4 years of Ad-hoc Interoperability layered on top of the specifications!

I strongly urge to read the entire paragraph “But we can talk to each other to make it compatible”, but I will rephrase the inevitable outcome of our current approach to fedi’s evolution:

The levels of interoperability of a federated application with any other implementations will depend on Luck.

To give just example here… Lemmy’s federation release was lucky to interoperate with Mastodon given the current state of the Fediverse, and later on the federation broke again, when the state evolved organically.

Note: There’s also Noel de Martin whho describes some different forms of interoperability in the article Interoperable Serendipity.

Fediverse needs Substrate to survive evolve

According to Cory Slep “Anything more than just sending and receiving information needs to be built on top of ActivityPub as an extension” (confusing here is that the AP spec is interwoven with its own extension: for microblogging). He calls the domains targeted by an extension a “flavor” and hence will create a “Fediverse of Flavors”. He says:

"[It is] preferable that each domain of the Fediverse builds one or more independent communities to standardize their specific solutions through the use of ActivityStreams extensions. […]

Unfortunately, when the ActivityPub protocol was published there was neither guidance nor support on how to build these communities, how to develop more specialized domains, or how to organize more rigorous behaviors around the data being shared. And there still isn’t."

These communities and their collaboration and cross-pollination I call “the substrate” of the Fediverse that must exist. The ‘substrate’ of an ecosystem such as the Fediverse are the people and processes and the artifacts they create that enable interoperability. The more decentralized, diverse and heterogeneous the ecosystem, the more need exists for a strong and healthy substrate to uphold it.

Dennis Schuber mentions XMPP as a example of a specification where 'Extensibility and Interoperability are closely guarded:

  • “they wrote a very strict base set of the absolute minimums required to build on XMPP” (IETF RFC 6120)
  • For definition of payloads they introduced XEPs, XMPP Extensions Protocols. And XEP-0001 describes the process for doing so.
  • Everyone can submit a proposal, and when following the rules, it is considered and when community finds it useful, adopted.
  • All extensions are available at a central location, and are managed by the XMPP Standards Foundation.

On the Fediverse a central body that manages these specifications is a sensitive subject. There’s resistence to any centralization.

The Fediverse at this moment unfortunately does not have a healthy substrate. It is evey developer for themself, look into codebases and issue trackers, try to find other developers to ask a question. W3C SocialCG and SocialHub community, initially very active, have lost momentum and relevance. Though a Fediverse Enhancement Proposal (FEP) process was started, it hasn’t really gotten underway.

Cory Slep formulates two problems to overcome:

“The first problem is getting different ActivityPub communities to self-organize to solve their own domain problems, build their own specifications on top of ActivityPub, and hold each other accountable at sticking to those specifications rigorously. A second problem is discoverability of these specifications for fellow developers without having a centralized organization hosting a registry or dictating a process. The cost of extending ActivityPub should always remain as close to zero as possible for numerous philosophical and moral reasons that I don’t want to get into here.”

Without a substrate the only way for a project to move forward is by Ad-hoc Interoperability. And once an app is successful there are less incentives to dedicate to substrate formation. There are already enough things to take into account to keep a FOSS project going.

Solutions for substrate formation

And in his article Cory poses an idea that really delights me, as it corresponds to directions I was also thinking in:

"Why not build on top of ActivityPub to solve both of these problems?

First, we as a community create an ActivityStreams extension and define behaviors to document the specifications built on top of ActivityPub. We now have a way to store ActivityPub extension specifications in the Fediverse itself. Next, building an application server that understands this extension and can federate it over ActivityPub means anyone can simply host an instance, allow users to register, build its community, and be free to draft new specifications on top of ActivityPub. Since this data is then federated by ActivityPub itself, it is reachable by both technical and non-technical users. There is neither a central authority nor a central registry as each community is responsible for its own ActivityPub specifications. This is the democratization of ActivityPub across all domains. People cannot have their ActivityPub extension specification censored. As long as the core ActivityPub protocol as currently written is followed, federation compatibility naturally leads to new domains and apps implementing them."

I proposed something similar a while ago in: Improvement to FEDERATION.md convention: Murmurations.

Note that these solutions are only a first start. It allows people to get informed of specifications and documentation, but in future more more machine-readable mechanisms are also needed, such as e.g. Capability Discovery and Compliance Profiles.

This does nothing to address the social aspects of substrate formation. Objectives are to:

  • Lower the barrier to participation as much as possible.
  • Automate as many of the chores as possible.
  • Highlight the incentives and win-wins of participation.

Related to these social aspects, in his last article Dennis Schuber highlights a point I highly agree with: Overall those developing the Fediverse are way too tech-focused! Note that the Social Coding Movement was started to address this:

Social Coding

Create solutions that people love. Social Coding is participatory free software development that is inclusive to all people and focused on addressing real human needs. Ideation, design and development is a collective and social effort.

As for positioning the Fediverse, I am personally promoting the idea of conceptually creating a more human ‘abstraction layer’ on top: the Peopleverse. Where the social interactions that are modeled are key, and not all the technical jargon that is now in our ‘elevator pitch’.

(Image credit: creativity103.com, “angled metal tracks on an electronic circuit board”, CC-BY)

2 Likes