API | Substrate

Our API design principles

The Substrate API consists of a carefully curated, well-abstracted library of components: nodes.

Nodes

Nodes (opens in a new tab) are designed following these principles:

High-level nodes like ComputeText provide an accessible, general interface for common operations.

For example, ComputeText is for invoking a language model with text (and optional images) and receiving text.

High-level nodes default to using a low-cost smaller model. As we release more capable models, we will automatically upgrade high level nodes to use the best option at equivalent cost. You can pin a specific model using the model (opens in a new tab) parameter.

The function of each node is focused in order to simplify inputs and outputs.

Most language model APIs follow the OpenAI completions (opens in a new tab) format, with output text nested inside an array of choices objects. But advanced usage patterns should never make the simple pattern more complex.
On Substrate, the output of ComputeText is simply text.
To generate multiple output choices for a prompt, use MultiComputeText.
To process multiple prompts in batch with higher throughput, use BatchComputeText.
The Multi and Batch pattern is used throughout the library.

Low-level nodes expose additional implementation-specific parameters.

When there is no clear unifying abstraction, only a low-level node is provided. For example, StableDiffusionXLControlNet is a specific way to generate images structured by another image.
When there is only one available implementation, only a high-level node is provided. For example, UpscaleImage only upscales images using a tiled control net method. When we add additional upscaling methods, we'll introduce low-level nodes.

If you are using a high-level node pinned to specific model, you can pass any of the pinned model's parameters through the high-level node:


ComputeText(
    prompt="tell me a story",
    model="Llama3Instruct8B",
    presence_penalty=2,
)

We plan to introduce a higher level layer distilling common subgraph patterns, and a lower level layer that enables disaggregated inference and deeper exploration.

Philosophy

Unix is one of the most enduring environments for programming, in large part due to its elegant abstractions grounded in a simple modular philosophy:

Make each program do one thing well.

Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information.

Substrate is grounded in the same modular approach, and these core beliefs:

Enduring abstractions and infrastructure must be built with each other in mind.
Complexity must be progressively revealed.

Mixture of Agents Box