Idea: Federated Supply-Chain Packaging

In chat @CSDUMMI pointed to the ongoing discussion on new PyPI rules that require “critical” packages to use 2-factor authentication (2FA) or be expelled from the index. Discussion relates to this blog post and response to it. Passing just for background context:

Problem

FOSS project that are used as a library dependency by other projects, likely leads to additional responsibilities and requirements. Packaging them for use in the supply-chain by package managers may come with a set of rules relating to quality-assurance (QA) and security. There’s a need for packages to be vetted on a continuous basis (i.e. throughout the release cycle), so others can use them safely.

These requirements come as an extra burden for the FOSS maintainer. It is a social obligation to comply with them in order to publish to a package manager service. Such service has its own social obligation to be a trustworthy source of packages, and - beyond social - may also offer stronger guarantees for those who depend on them. In general…

James Bennet: “Open source really is a social activity, and as a social activity it comes with responsibilities and benefits for multiple parties in multiple directions.”

The issue that arises is that “It’s not the fault of the creator that their creation became popular.” as Armin Ronacher states. They may be overburdened already, volunteering their time creating the project. Or they may not be interested to take all these responsibilities. And imho that is their right too, part off the “freedom” in free software. Whether they comply with the rules and implicit obligations that come with publishing under an open source license, and to what extent they do so, is their own choice. But it can have consequences.

Let’s instead of talking about “project maintainer” talk about “social coders” in general. This is a minimum requirement:

A social coder has the bare minimum responsibility to set expectations.

(Note that Expectations are key to the alternative technology adoption models discussed in Fixing the Fediverse Adoption Lifecycle.)

If you have a hobby project and don’t really care about security implications, then you should clearly mention that. You may still want other people to use your project on those conditions. And here’s another problem…

Centralized package managers

Most package managers are centralized. Think e.g. NPM (owned by Microsoft) and PyPI too (mostly by volunteers). Getting your package out means you MUST use them. And comply to their demands. This can be a slippery slope, as the ‘rulebook’ can grow and grow, e.g. by corporate demands.

Solution

So how can we improve ecosystem participation as provided by Package Manager Services?

  • Make it as easy as possible for project maintainers to comply to rules for inclusion in a package manager.
  • Share the burden as much as possible between all the stakeholders that benefit from using the package.
  • Allow alternatives to centralized package managers to emerge that come with different policies and expectations.

Inspiration

  • Jazzband: A collaborative community to share the responsibility of maintaining Python-based projects.
    • An example of a social community organization the share the burden of packaging and vetting.
  • cargo-vet: Integrated supply-chain security for Rust provided by Mozilla to audit dependencies by trusted sources.
    • An example of a technological solution to help lower the barrier to collectively foster a trusted ecosystem.

Federated package management

There’s a ton of “social” in all of this, and it should be clear that Fediverse social networking can play a role in this solution. Mozilla Cargo Vet already comes a long way to sketch how a decentralzed solution looks like from a technical perspective. Reading from the How it works page, the project is positioned to (highlight mine):

Cargo Vet: Most developers are busy people with limited energy to devote to supply-chain integrity. Therefore, the driving principle behind cargo-vet is to minimize friction and make it as easy as possible to do the right thing. It aims to be trivial to set up, fit unobtrusively into existing workflows, guide people through each step, and allow the entire ecosystem to share the work of auditing widely-used packages.

On top of that it decentralizes the registry. Where the description (and the entire project, for that matter, which is quite technical oriented) isn’t very clear on is how the Auditing Workflows are done from a social perspective. But here’s where the Fediverse shines.

The project mentions local workflows or those handled by SourceGraph, a commercial SaaS vendor. It may well be that this project leads to a limited number of rather centralized ‘vetting hubs’.

Packaging and Social Coding FSDL

Packaging has been added to the Free Software Development Lifecycle and so has Supply-Chain Packaging as a sub to that. Social Coding aims to find best-practices for the FSDL along with tools that support them, and in particular to wield the Fediverse as a technology base in support of the social aspects of software development. Let’s brainstorm some Fediverse supporting features…

Brainstorming the solution

The following is just brainstorm. In your feedback don’t criticize but refine, improve, expand on these early ideas and musings :slight_smile:

  • Anyone can spin up Software Packaging Communities and federate with other ones.
  • A community can target one or more programming languages, and multiple packaging services.
  • Packaging communities can sit as proxies between social coders / fedizens and (official) package managers.
  • Communities can set their own vetting rules, publish them as Policies, and provide Guidance for Auditers.
  • Consumers of packages are free to choose compliant packages based on Policy of their preference.
  • As Auditers review packages, their reviews are vetted just as well on various quality aspects.
  • There may be a reputation system aimed to bring forward most trustworthy reviewers, weed out the bad apples.
  • As FOSS project Registries build up, they contain a lot of project metrics that open up many different use cases.
    • You can subscribe to registries, build up directories, AlternativeTo sites, FOSS developer profiles, etc.

(Image credit: Pixabay via Pexels)


Link to this topic posted to the Fediverse and to the Lemmy Fediverse Futures community (that will be part of Social Coding as well).

Just some examples that just popped up…

  • On Hacker News there’s a question now: How do you search for products / apps given a list of requirements?

    • There’s a comment: “Wikipedia already has great tables for comparing software […] but since many products aren’t comprehensively tagged, you’ll miss a lot if you use the filter (something that crowd sourcing could solve).”
    • Project characteristics as Linked Data would facilitate this use case, and they can be part of / extension to federated package management co-creation workflows.
  • Using Open Badges as proof of qualification for packages. A badge-based solution is itself a candidate for federation.

    • Show these badges on your repo’s README, share them on your Resume, etc.
    • With the Open Badges technology parts of the ‘reputation management’ requirements can be covered.
    • Note: Doug Belshaw on fedi, is an absolute expert in this field, who could provide more insights here.

In this should also mention the ability to make distinctions by the type of actor involved: Person / Group / Organization.

  • Having respectable organizations like FSFE, Linux Foundation, etc. endorse Auditors or do the vetting, would lend them automatically more credibility

As a developer I’ve used a dozen packaging systems over the past few decades. What puzzles me most is that although it is a really simple problem with few concepts, it’s been fragmented and partially implemented for the longest time. Just think of a simple problem: cyptographically signed packages is a basic requirement that is not implemented in most repositories. Or another, more complicated but for which there are known solutions: resolving dependencies between packages is only partially implemented on most packaging systems.

Maybe federation will help packaging systems to unite but I honestly have no clue why or how. It’s going to be an interesting evolution to watch.

1 Like

SPDX specification

Cross-linking to the ISO open standard for creating Software Bill Of Materials to help analyse the software supply chain. When implementing this idea especially the existing open source tools should be looked at.