Doing <thing> when <trigger> occurs is quite useful. Listening for <trigger> is computationally inexpensive but requires something to always be on. Doing <thing> may be computationally expensive but only requires on-demand availability.

Substrate Lambdas aims to provide a self hosted FaaS (function-as-a-service) platform.

Repository Goals

(1) Provide developers with a platform to rapidly develop & easily deploy lambda style applications which follow a standardized design pattern & set of tools. One important tool will be the ability to launch remote workers across various cloud providers.

(2) Provide users with platform to easily plug-n-play various "apps" or "plugins" built by developers

(3) Be as lightweight as possible

(4) Be as readable as possible for a layperson (specifically app code, but also core code where possible). Code is truth— no process is trustless unless you personally can read the code.

Features

Syntactic Sugar

Substrate Lambdas leveragees Typescript's rich type system to build an elegant app development experience resting on top of papi. When designing the interface, our goal is to allow app developers to easily specify their applications intentions with:

1. minimal required prior knowledge of blockchain concepts
2. minimal required code for an app's specification (without introducing rigidity or obfuscating what's going on under-the-hood)

import { App, Observables } from "@lambdas/app-support";

const description = `
Description of how this app works & what it does
`;

export default App(description, {
    watching: Observables.event.SomeChainId.SomePalletId.SomeEvent(),
    trigger(payload, context) {
        // do some filtering . . .
    },
    lambda(payload, context) {
        // do something upon triggering . . .
    },
}, { /** More routes across many chains . . . */ });

Observables is a powerful entry point which stitches together TypedAPI's across all chains with available descriptors, with additional features. It

• Acts as a point of discovery for what is available across the polkadot ecosystem.
• Captures real-valued meta data about an application (such as chain dependencies of this application)
• Seemlessly makes relevant type information available to developers
• .all(): Allows specifying Blanket routes on events, which observe many events at once. For example, we may specify Observables.event.Polkadot.Balances.all()to watch all events within Polkadot's Balances pallet, or Observables.event.Polkadot.all() to watch all Polkadot events, with a single route.
• narrowPayload: Convenience function for branching to custom logic for some subset of watching when watching multiple observables in a single route. example:

App(description, {
    watching: [
        Observables.event.polkadot.Balances.Deposit(),
        Observables.event.polkadot.Balances.Withdraw()
    ],
    lambda(payload, context) {
        // `typeof payload` is either of `Balances.Deposit | Balances.Withdraw`
        if (narrowPayload(payload, this.watching[0])) {
            /**
             * Now `typeof payload` is specific to the 
             * `Balances.Deposit` event in intellisense
             * 
             * Do custom transformation . . .
             */ 
        }

        // Shared logic . . .
        aggregate(payload);
    },
});

You can refer to the list of supported known chains.

Workers

A core feature of Substrate Lambdas is the ability for apps to launch remote jobs in response to on-chain events. We provide a simple unified interface for app developers to launch jobs across a variety of cloud providers & server types.

Platform	Type	Provider	Payment Method	Development Status
Vast.ai	GPU	Open marketplace	Credit Card ☹️	⚠️ Experimental
OctaSpace	GPU	Open marketplace	Ethereum 🤠 (Maybe native Polkadot ecosystem support in future? 🥰)	🔄 To-do
Libcloud	CPU/GPU	Supports more than 50 mainstream cloud providers	Credit Card ☹️	🔄 To-do
*Self	CPU/GPU	Self-hosted	--	🔄 To-do

App Registry

Substrate Lambdas will provide an app registry for developers to publish their apps to. The first iteration of the registry will simply be a HuggingFace repository with a folder for each app. Developers will upload their apps using the dothome CLI. This will allow developers to easily publish/modify their own applications while restricting access to modify unpermissioned sections of the repository hosting the registry.

Why HuggingFace? Well— it's git configured for handling big data... and gives you A LOT of free storage 🤫. Great for dumping artifacts.

CLI: dothome

Note: this command table is in the "idea simmering" stage. Largely just an aggregation of living notes that will converge to a more sensible/stable idea in the coming months

Command	Sub-command	Input	Description
app	list		List all apps
	status	<app-name>	Show app status
logs		<app-name>	View logs
worker	list		List workers
	connect		Connect to worker
	kill		Terminate worker
overview	apps		Overview of apps
system	account		System apps info
	users		System apps info
	health		System apps info

Robust Platform

TODO! extrapolate...

• Metadata monitoring: while Substrate Lambdas is running, we actively monitor for metadata updates for all chain dependencies
  • if we can, same kind of monitoring where-ever possible, like app updates (although these updates would need review before being applied). But this is important for detecting breaking changes and needing to shut down apps until updates are installed.

Quick Start

All apps in src/apps will be run. Any apps that you would like to disable, rename the folder with a prefix of _. If you would like to run a custom app, create a new app folder & follow the specification defined in the Apps section.

Depending on the applications running you will have to define a .env file in the root directory.

# election-watch
EMAIL=<your-gmail>
PASSWORD=<your-gmail-appkey>

# polkadot-election-dataset-aggregator
HF_TOKEN=<your-huggingface-token>
REPO_NAME=<your-huggingface-repo>

Local

Prerequisites: Node.js & Python

Just run setup.sh and then npm start.

Fly.io

We include a Dockerfile & fly.toml for easy deployment to fly.io. First install the CLI tool:

brew install flyctl

The first time you run this, you will be prompted to login and connect your credit card. I think that I've set up the settings s.t. you won't be charged anything.

# launch
fly apps create substrate-lambdas
fly deploy
fly scale count 1 -y # scale downn to single node

# shut down
fly apps destroy substrate-lambdas -y

Run tests

Our testing suite assume the descriptors polkadot, polkadot_asset_hub and rococo_v2_2 are available.

Unit tests can be run with npm run unit:test. To run integration tests, first run npm run testup in one terminal instance, which will spin up the necessary chopsticks mock chains. Then you may run npm test which will run all unit & integration tests.

Create an app

Applications are expected to be defined in src/apps/<app-name>/index.ts with the default export defined with a description & any number of routes, using the given App builder. (Example)

In future iterations, the core of Substrate Lambdas will be available as a package & apps will be defined in their own independent projects— decoupled from the core code.

built on papi