Background Removal

An introduction to other image generation techniques

Red chair in a room

Remove the chair

Let's build a workflow that removes the foreground object from an image. The pipeline:

Generates an image using GenerateImage
Removes the background of the image using RemoveBackground, creating a foreground mask
Erase content inside the mask using EraseImage
Fill in details using InpaintImage

First, initialize Substrate:

Python

TypeScript


from substrate import (
    Substrate,
    GenerateImage,
    RemoveBackground,
    EraseImage,
    InpaintImage,
    sb,
)
s = Substrate(api_key=YOUR_API_KEY)

Generate an image using GenerateImage.

Advanced image generation

For more control over Stable Diffusion XL, use StableDiffusionXLLightning (opens in a new tab).
Provide a seed to "pin" the initial noise in the image generation process – this can be a good way to experiment with subtle changes in your prompt.

Python

TypeScript


prompt="by edward hopper, a dark red chesterfield leather wing chair in a dark majestic room, pillars, celestial galaxy wallpaper",
image = GenerateImage(
    prompt=prompt,
)

Remove the background from the image, and turn the result into a "mask" (a black and white image). Without return_mask=True, RemoveBackground returns the foreground segment of the image.

Python

TypeScript


mask = RemoveBackground(
    image_uri=image.future.image_uri,
    return_mask=True,
)

Erase the content inside the mask using EraseImage. This produces a rough approximation of erasure, so we follow it with InpaintImage. (For deeper control of the inpainting process, use StableDiffusionXLInpaint.) Finally, call substrate.run with the terminal node of the pipeline, inpaint.

Python

TypeScript


erase = EraseImage(
    image_uri=image.future.image_uri,
    mask_image_uri=mask.future.image_uri,
)
inpaint = InpaintImage(
    image_uri=erase.future.image_uri,
    prompt=prompt,
)
res = s.run(inpaint)

Here's a sample result from this pipeline:

res.get(image)

res.get(mask)

res.get(erase)

res.get(inpaint)

Image generation Structured image generation