Deploy Stacks

Updated 3 months ago

This page shows how to create and deploy Koyeb Stacks to upload, process, and store your data. Stacks are composed of three core components:

  • Sources: Integrations to upload and ingest your data. i.e., Koyeb S3 API Gateway, Import from Dropbox, Import from AWS S3. Check out the catalog to discover all Sources. At the moment, only one source is available and activate on Stacks, the Koyeb S3 API Gateway. This integration exposes an S3-compatible API endpoint to let you upload your content.
  • Workflows: Pipelines executing a series of processing actions on the data. You can build your processing Workflows or use Workflow templates to start in minutes. Stacks can contain one or more Workflows. You can define when your Workflows or Workflow processing steps are executed using conditional triggers.
  • Stores: The place to store the data uploaded and processed by Stacks. By default, Koyeb provides managed storage space for simplicity and reliability. You can also connect third-party cloud storage services, i.e., AWS S3, GCP Storage, Azure Blob, DigitalOcean Spaces, and much more.

After you have deployed a Stack, you can start the upload and processing of your data using the Koyeb S3-compatible API endpoint.

There are four steps to deploy a new Stack:

  1. Add a Workflow template
  2. Give the Stack a name
  3. Configure the destination Store (optional)
  4. Deploy the Stack

Get started

On the Koyeb control panel navigation bar, click the "CREATE" button and select Stack.

Koyeb home with create menu expanded

You land on the Stack creation page.

Add a Workflow template

On the Create Stack page, in the create workflows section, click the "CREATE FROM TEMPLATE" button. Workflow templates are ready-to-use processing Workflow integrations implementing the most popular and requested processing pipelines to let you get started with ease. You can also compose your processing workflow from scratch.

The list of available Workflow templates appears showing for each template:

  • The Workflow template name.
  • The Workflow template description.
  • The provider of the Workflow template.

Stack new with workflow templates list expanded

Let's select the "Image optimizer" template. This template store all uploaded images in the folder original/, optimize the source image by reducing the image size while maintaining the same visual quality, and store the result in the optimized/ folder.

Once selected, the Workflow appears. You can click each step of the Workflow to change its settings. You can also add processing steps to the Workflow template.

Stack new with workflow template image optimized selected

Give the Stack a name

The Stack name is a unique name identifying your Stack and acts as the bucket name. It is used to interact with the Koyeb S3 API.

Configure the destination Store (Optional)

By default, Koyeb creates a Store for your Stack. In some cases, you need to use an existing Koyeb Store or connect a third-party cloud storage provider to act as your Stack Store.

To connect a Store, click the edit icon in the store section.

Stack new with edit store open

Select a Store integration

A drawer opens listing the integration available as a Store. In this documentation, I configure AWS S3 as a store for my Stack. The procedure for other Store integration is quite similar.

Configure the Store integration

Stack new with edit store s3 selected

To configure the Store, I need to define the bucket name and region to use to store the data processed by the Stack. Parameters may vary depending on the integration you select.

Deploy the Stack

We are now ready to deploy our Stack and start processing data. Click the "CREATE STACK" button to provision the freshly configured Stack.

Stack page index

You land on the Stack where are displayed various information about your Stack such as:

  • The Stack status: The status of your Stack. It can be DISABLED, ENABLED, LOCKED or ERROR.
  • The Stack Workflows and their status. It can be DISABLED, ENABLED, LOCKED or ERROR.
  • The Stack Store and it's status. It can be DISABLED, ENABLED, LOCKED or ERROR.
  • The list of execution: For each file uploaded to the Stack and processed by a Workflow, an execution containing: the file name and the associated execution status (PENDING, SCHEDULED, RUNNING, SKIPPED, SUCCESS, FAILED, CANCELLED) appears in the list. Execution are created when Stacks contain at least one Workflow and allow you to track the processing execution status for each file processed by a Workflow.

You can now start uploading and processing your files using the Koyeb S3-API. Check out the documentation to integrate the Koyeb S3-API with existing SDKs and CLI tools here.