Solidground: WASM Component-based development (wiki)

Solidground
#1

WASM Technology survey

This wiki contains may links to WebAssembly related resources that were encountered over time. And the wiki is updated as new resources are found. It is NOT meant to give an exhaustive overview of the field and its current status, but just serves to inspire on some of the ways that Wasm is used in-the-wild.

Open standards: WASM, WASI, Component Model

Background reading:

The introduction of WASM to the server, the cloud, and the edge is more far-reaching than initially apparent. It opens a completely new paradigm to Component-based architecture.

  • Fine-grained plugin model. Modularity. Nano processes. Polyglot.
  • New software delivery model. Beyond containers.
  • Security-first design. Trustless computing. Capability-based security.
  • Performant, size/memory efficient, suitable from micro scale and upwards.
  • Focus on Developer Experience, ease of use. Minimize stack/infra maintenance.
  • Interoperability, portability. Based on open standards (e.g. Component Model).

Terminology popularized by use cases & projects based on this approach (need not be WASM-related):

What say the experts?

In a recent publications (Google Doc turned PDF below), Ian “Hixie” Hickson, editor of the HTML specification for 10 years, in January 2023 criticises the use of hypermedia for intricate web applications, and proposes an alternate approach based on Wasm (defending it on Hacker News), among others:

By providing low-level primitives instead, applications could ship with their own implementations of high-level concepts like layout, widgets, and gestures, enabling a much richer set of interactions and custom experiences with much less effort on the part of the developer.

As it happens, among the many high-level primitives, a few low-level APIs have been created and are available on the web today:

  • WebAssembly (also known as Wasm) provides a portable compilation target for programming languages beyond JavaScript; it is being actively extended with features such as WasmGC.
  • WebGPU provides an API (to JavaScript) that exposes a modern computation and rendering pipeline.
  • ARIA (Accessible Rich Internet Applications) provides an ontology for enabling accessibility of arbitrary content.
  • WebHID provides an API (to JavaScript) that exposes the underlying input devices on the device.

This document proposes to enable browsers to render web pages that are served not as HTML files, but as Wasm files, skipping the need for HTML, JS, and CSS parsing in the rendering of the page, by having the WebGPU, ARIA, and WebHID APIs exposed directly to Wasm.

Ian-Hickson_Towards-a-modern-Web-stack_2023.pdf (99.5 KB)

(Tooted on the Fediverse in this post)

Solidground specific observations

[Paradigm] Control the Supply Chain

“WebAssembly Everywhere” represents a paradigm shift which allows fine-grained composition of services, that aligns perfectly to the idea of App-Free Computing. Control of the supply line, dependency management, new ways of packaging and deployment of solutions and a primary focus on Developer/DevOps Experience to bring productivity gains, represents a powerful value proposition.

There’s a lot of attention for compliance and open standards, probably driven by the urgent Need and focus on Supply Chain control. An area where Free Software is firmly in the rear guard, which represents a big threat. Prosperity Guilds is exploring the service Software Supply Chain where Solidground can add value.

Wrt packaging and deployment, the IT practices have gone crazy complex in recent years. The paradigm shift intends to address this low-hanging fruit of productivity gains. WebAssembly allows more fine-grained deployment (terminology found: ‘micro-microservices’, ‘nanoservices’). Docker is recognizing this:

Regarding software delivery model consider this diagram:

Diagram showing deployment models from bare-metal to Distributed Wasm
Diagram showing deployment models from bare-metal to Distributed Wasm1199×667 131 KB

In the diagram, where Cosmonic position themself is where the general WASM-based delivery model is (but without the Actor Model, which is added value offered by wasmCloud and Cosmonic).

Component registries

Related to Supply Chain above, but also Groundwork plugins and App-Free Computing needs…

warg is an in-development open source registry protocol for Wasm packages (components or ‘world’ interfaces). warg provides an index allowing everyone to refer to federated namespaces of Wasm packages without being opinionated about how package content is stored and hosted. Warg is:

  • Component Model oriented.
  • Federated & highly interoperable.
  • Content storage & delivery agnostic.
  • Advancing supply-chain security.

This protocol is still under development.

[Paradigm] Small is Huge

Going from a philosophy that “Small is Huge” → Everything works better when it is still small, and complexity comes with size. Solidgrounds wants to adhere to Small Technology principles.

Though there’s a lot of ‘cloud-native’ and ‘serverless’ terminology, the “Wasm Everywhere” paradigm facilitates any scale and any edge devices. Whether services run on a microcontroller IoT devices, mobile phone, web browser, VPS or large cloud clusters… it is all supported.

Opportunity: A niche exists for FOSS ecosystems that focus on ‘smallness’ in their offerings.

What all the current vendors and tech companies exploring Wasm have in common is their love for ‘high scalability’. This is a flaw and a weakness that we in Free Software movement can benefit from. Our scale comes from Decentralization.

Opportunity: Polyglot being intrinsic to “Wasm Everywhere” fits fragmented grassroots efforts.

@Ryuno-Ki mentioned the importance for guild members to have enough alignment in technology use, in order offer most value in services exchange. WebAssembly can be the common glue technology, allowing any member to use their own preferred techstack.

  • SQLite gets official Wasm support with help from VMWare’s WasmLabs (see: HN discussion).

[Concept] Capability-based programming

WASI has a Capability-based security model. It is a bit confusing how this relates to wasmCloud / Cosmonic projects. This HN comment asks a question, answered by OCaps expert.

In wasmCloud the cryptography is handled by the tooling. For Capabilities you need Claims that are packaged as a JWT. You define the namespaced capabilities yourself, and they become part of the wasmcloud.yaml of your actor app. You have fine control of what it means when a capability is present, as the capability is defined by its interface (contract based programming). If you expose a key value store to your app, in production that can mean any KV store implementation is invoked, shielded behind the interface.

[Concept] Visual component composition

WebAssembly Components are a proposed extension on Wasm, that allows Wasm Modules to be encapsulated, and become composable and adaptable. The interface, imports/exports and execution environment can be described as “WIT” Worlds, allow Interface-based Programming (Contract-based programming).

Fastly.com presented at the Cloud Native Wasm Day Europe 2023 a demo featuring a visual drawing canvas, allowing WASM components to be inserted e.g. from a component registry (a federated, distributed namespace of Wasm components), connected together with lines to satisfy import and then exported as a .wasm file that constitutes the composition.

The wasmbuilder.app demo is online, and Apache 2.0 project on Github in Rust + Typescript.

[Vendor] Cosmonic WebAssembly PaaS

“From a simple idea to running your apps anywhere, at scale, in minutes, Cosmonic is the lightweight, low-boilerplate platform that radically simplifies the development lifecycle.”

  • “From Sketch to Scale”
  • “Any Cloud, Any Edge, At Any Scale”
  • “Untangle Dev and Ops”

Cosmonic is a leading early adopter vendor (chairing Cloud Native Foundation). Co-maintainers of wasmCloud, which they brought into the Cloud Native Foundation, and improving compliance for. Their Paas is based on wasmCloud, but they also have several independent OSS projects. Their philosophy is to be (permissive) open-source.

Key aspect in wasmCloud (which has both a Rust and Elixir impl) and Cosmonic is to “separate business logic from non-functional requirements (NFR’s)” … which nears same objective of Clean Architecture pattern’s infra layer, where Ports & Adapters are “Capability” abstractions.

Note: Vendor Deis Labs works on Spiderlightning CLI to support common Capabilities, defined as WIT and tested against implementations.

wasmCloud uses a bunch of standards: Open Container Initiative (OCI), Open Application Model, Cloud Events, NATS networking and Open Telemetry.

Note: Cosmonic is also offering visual modeling in their tooling:

https://cosmonic.com/assets/medias/untangledevops-985a88a25c08ffa08e89f4ac185de047.mp4

[Project] Cosmonic Concordance: Actor-based CQRS/ES

Recently Cosmonic created Apache-licensed Concordance project in Rust, based on wasmCloud. It is opinionated in that the core of the CQRS/ES model is entirely side-effect free. From the README:

Concordance allows you to reason about your application using the fundamental building blocks of event sourcing:

  • Aggregates - WebAssembly components that derive state from an event stream, validate incoming commands, and emit events in response to those commands.
  • Projectors - WebAssembly components that project read-optimized data (aka materialized views) based on streams of incoming events
  • Process Managers - Known as sagas in other paradigms, a process manager is the inverse of an aggregate: it processes inbound events and starts, advances, and stops processes by emitting the corresponding commands.
  • Stateless Event Handlers - used for entities such as notifiers and gateways, you can build WebAssembly components that receive events on a stream and then execute whatever you choose for your business logic.

“Constraints liberate, liberties constrain” - Runar Bjarnason

This quote is a foundational principle behind this event sourcing library. You’ll find that aggregates are unable to do anything other than return a list of events. They are unable to produce side-effects, which is a good thing™. Likewise, process managers cannot produce side-effects, they can only return a list of commands. Side-effects are explicitly forced into the perview of notifiers (stateless event handlers).

Concordance looks to be independent of the Cosmonic PaaS, but this product is also big on event sourcing, and the CTO Kevin Hoffman (on the fedi, and writing about fedi) is writing the book Practical Event Sourcing (will be available under Resources page).

Project is under heavy development. The Bankaccount example app for instance is stand-alone (had Concordance classes embedded). Article on how WIT Worlds can shield from framework impl:

[Concept] Cloud Development Environment (CDE)

Bye bye, forge and git… CDE’s are on the rise and offer one-stop-shop developer experiences from the browser. This is going to be a big paradigm shift. Vendors all around are jumping on the fact that software development wrt configuring, deploying, maintaining the infra and cloud have become crazy complex.

WebAssembly-Everywhere offers a path to solving the infra and tooling complexity.

(FOSS is lagging behind. One-stop-shop platforms poses a threat to Free Software if it doesn’t catch up. The #forgers-guild aims to go beyond “forge federation” to “forging free software”, decentralized and scoped to Social Coding FSDL, and bring unique strengths of Free Software movement into play.)

[Fediverse] DDD, AS/AP, Solid, DID/VC (OMG :scream: )

Quite unexpected finding of Naamio app for the Fediverse, based on Wasm, and developed by (mysterious) cooperative Omnify Foundation. Uses ActivityPub, DIDs, Solid, and Verifiable Credentials.

Diagram "Modern Data Mesh Revisited", Domain-Driven Service Development
Diagram "Modern Data Mesh Revisited", Domain-Driven Service Development1207×586 69.5 KB

Note: Uses combination of Wasm (fine-grained composition) and gRPC (system orchestration).

[Concept] Universal plugin models

The BSD3-Clause project Extism: The Universal Plugin System right now offers integrations for 15 languages and the web browser. Their goal is to make all software programmable, and their plugin system lends sandboxed polyglot support for that. Integration requires a Host SDK and then Plugin Development Kits (PDK’s) in your language to quickly write plugin code.

Example: Elixir Game Server + Extism plugin integration.

GameBox conceptual architecture diagram
GameBox conceptual architecture diagram1747×816 45.8 KB

Elixir’s actor model handles real-time user interactions playing in a “game arena” via PubSub in Phoenix LiveView, while the games can be written in any language that’s supported by Extism.

[Concept] Erlang/OTP concurrency on Rust

Lunatic Solutions offers a wasm-based Erlang/OTP-like actor model implementation in native Rust.

Lunatic combines the fault-tolerance and massive concurrency of Erlang with the capability-based security of WebAssembly, it creates a powerful programming model.

Lunatic is a single executable and runs on Windows, macOS and Linux. It spawns lightweight processes from WebAssembly modules that have their own heap/stack. These processes are preemptively scheduled on a multi-threaded executor.

[Concept] Rust-based Web UI

Great comparison of emerging Rust-based GUI approaches (see also: HN Discussion, 268 comments)

Leptos: Build Fast Web applications with Rust is a reactive web framework most similar to SolidJS and Sycamore (Rust) and with inspirations from other popular libs. See the Online Book (In progress).

Looks properly usable. Following demo shows some development with TailwindCSS in Rust:

While Leptos has the more modern approach to Web Design, there’s a more popular web framework in Rust that supports WebAssembly, which is:

  • Yew: A framework for creating reliable and efficient web applications.

Yew is set up more as a React-like framework that is diffing the DOM.

[Innovation] Makepad Modern GUI’s in Rust

Though related to the UI topic above this one, Makepad deserves a place of its own. A seriously impressive work in Rust where complex GUI’s are defined by a Figma-like DSL with background code support. It is an effort to bring Coders and Designers closer together. Wasm/WebGL is one of their compilation targets.

:point_right: This hints at a paradigm beyond the browser straitjacket, moving away from Web 2.0!

Seriously consider watching this video, where the slides are done with Makepad and run real-time multiple applications within it at 120 fps…

One very cool aspect re:Floorplanner is that with Makepad both the IDE and the GUI design can travel within the app itself and be live-coded and changed. Downside: No a11y support (I filed an issue).

[Bounded context] Floorplanner Diagramming

Bounded context diagram for Social Experience Design

I found one Yew project (no longer maintained), called yew-flow (see demo), which was an attempt to port React Flow - best candidate for integration in Floorplanner thus far - and offer that as a Wasm module.

This may not be the best approach… I am thinking more in the line of: Code that wraps any NPM project into a Wasm module or - better - component. A WIT-based component interface would be exposed and allow for different diagram type defitions to be fed to the component, and domain-specific diagram output to be fed back.

[Innovation] Freenet Rebooting

(Update: Freenet.org is not really a reboot, rather a hostile fork by the founder against the community)

Freenet.org started in 1999 and was quite popular for a time and still going today (Wikipedia). In 2023 the community started a new project called Locutus in Rust (Matrix chatroom):

Freenet 2023, also known as Locutus, is a fully decentralized, peer-to-peer network and a drop-in replacement for the world wide web. It operates as a global shared computer, providing a platform for sophisticated decentralized software systems. Locutus allows developers to create decentralized alternatives to centralized services, including messaging, social media, email, and e-commerce. It’s designed for simplicity and flexibility and can be used seamlessly through your web browser. The platform’s user-friendly decentralized applications are scalable, interoperable, and secured with cryptography.

Locutus is a decentralized key-value database. It uses the same small world routing algorithm as the original Freenet design, but each key is a cryptographic contract implemented in Web Assembly, and the value associated with each contract is called its state. The role of the cryptographic contract is to specify what state is allowed for this contract, and how the state is modified.

Freenet (or similar concept) might have a role in Prosperity Guilds as a backbone (just an idea).

[Showcase] Wordpress Playground

Automattic launched an official Wordpress Playground that runs a full Wordpress server entirely in the browser based on WebAssembly. Discussed on HN in this submission.

The operating system of the web – on your device

Web browsers are just the beginning – WordPress Playground works in Node.js, mobile apps, VS Code, and more. Whether you want to build a note-taking application for mobile devices, an automated testing environment, or demo a live WooCommerce store on your site – Playground will support you.

How is this possible? Playground uses the latest, cutting-edge technologies to make the key WordPress dependencies work in JavaScript.

[Project] VectorVisor in Rust

VectorVisor is a vectorizing binary translator for GPUs, designed to make it easy to run many copies of a single-threaded WebAssembly program in parallel using GPUs.

"GPU programming models offer a level of abstraction that is either too low-level (e.g., OpenCL, CUDA) or too high-level (e.g., TensorFlow, Halide), depending on the language. Not all applications fit into either category, resulting in lost opportunities for GPU acceleration.

VectorVisor is a vectorized binary translator which remedies this issue by taking in existing single-threaded WebAssembly programs and running many copies of them using a GPU."

[Product] Nginx Unit Universal Web App Server

NGINX Unit is a lightweight and versatile application runtime that provides the essential components for your web application as a single open-source server: running application code (including WebAssembly), serving static assets, handling TLS and request routing.

Among key features is WebAssembly support that allows running any Wasm modules/components on the server.

[Product] Ambient Game Engine

Game engine built with Rust, WebAssembly and WebGPU. Our WASM use is innovative; it is being used as both a sandboxed runtime for user code, and as a way to run the entire Ambient runtime in the browser. This, paired with our ECS data model, enables a highly modular architecture that allows other developers to make isolated packages (code, 3D models, tools, etc.) that they can add as a mod to your game.

The use of wasm in combination with the Entity Component System may be very flexible in App Free Computing component-oriented social experiences (e.g. modular UX/UI).

[Product] Enhance WASM: Vendor-neutral server-side rendering Web Components

Enhance WASM is bringing server side rendered web components to everyone. Author your components in friendly, standards based syntax. Reuse them across multiple languages, frameworks, and servers. Upgrade them using familiar client side code when needed.

Read: Introducing Enhance WASM. The idea to package SSR-rendered WebComponents into WebAssembly modules is appealing.

[Project] FluentCI Engine: Programmable CI/CD engine without Containers

FluentCI Engine is a programmable CI/CD engine that is designed to be simple, flexible, and easy to use. It is supposed to run on the host machine without containerization or virtualization, and it is designed to be used with Nix, Pkgx, Devbox, Flox, Devenv, EnvHub, Pixi and Mise.

MPL-2.0 licensed. Implemented in Rust/Typescript and using WebAssembly-based plugin system, based on Extism.

Features

  • Simple and easy to use
  • Flexible
  • No containerization or virtualization
  • Built-in support for Nix, Pkgx, Devbox, Flox, Devenv, Envhub, Mise and Pixi
  • Cache support (backends: local, S3, GCS, R2)
  • SDK for writing pipelines in TypeScript, see @fluentci/sdk
  • GraphQL API, see API Documentation
  • OpenTelemetry tracing
  • Plugin system in WebAssembly, see examples

Code snippet in Rust invoking a pipeline via the plugin system
Code snippet in Rust invoking a pipeline via the plugin system1115×520 219 KB