Log Exporter

Elasticsearch

To export your logs to an Elasticsearch  instance, create a service using the docker.io/koyeb/log-exporter image with the <VERSION>-elastic tag:

docker.io/koyeb/log-exporter:<VERSION>-elastic

You can optionally click the Deploy to Koyeb button below to configure the log exporter to ship to an Elasticsearch instance with much of the configuration pre-filled. Update the environment variables to correspond with your values and choose any other customizations before deploying:

Currently, only basic authentication is supported.

During the deployment, set the following environment variables to configure forwarding to Elasticsearch:

• KOYEB_SERVICE: The name or ID of the Koyeb Service to export logs for. This takes the form of either <APPNAME>/<SERVICE_NAME> or <SERVICE_ID>.
• KOYEB_TOKEN: A Koyeb access token generated in the API section of your Koyeb organization settings .
• ELASTICSEARCH_ENDPOINT: The HTTPS URI where your Elasticsearch instance accepts incoming messages.
• ELASTICSEARCH_USER: The Elasticsearch username.
• ELASTICSEARCH_PASSWORD: The Elasticsearch password. We recommend storing this as a Secret.

Additional resources:

• Elasticsearch image tags on Docker Hub 
• Elasticsearch directory in the log exporter repository 
• Vector sink documentation for Elasticsearch 

Splunk

To export your logs to Splunk , create a service using the docker.io/koyeb/log-exporter image with the <VERSION>-splunk tag:

docker.io/koyeb/log-exporter:<VERSION>-splunk

You can optionally click the Deploy to Koyeb button below to configure the log exporter to ship to a Splunk instance with much of the configuration pre-filled. Update the environment variables to correspond with your values and choose any other customizations before deploying:

During the deployment, set the following environment variables to configure forwarding to Splunk:

• KOYEB_SERVICE: The name or ID of the Koyeb Service to export logs for. This takes the form of either <APPNAME>/<SERVICE_NAME> or <SERVICE_ID>.
• KOYEB_TOKEN: A Koyeb access token generated in the API section of your Koyeb organization settings .
• ENCODING_CODEC: The message encoding to use. Defaults to json. Supports all Vector codec values  except for avro and csv.
• SPLUNK_ENDPOINT: The URI where the Splunk instance accepts logs.
• SPLUNK_TOKEN: The Splunk HTTP Event Collector (HEC) access token  used to authenticate to the Splunk collection endpoint.
• SPLUNK_INDEX: The index in Splunk to save the logs to.
• SPLUNK_SOURCE_TYPE: The Splunk sourcetype used to identify the log lines. Defaults to koyeblogs

Additional resources:

• Splunk image tags on Docker Hub 
• Splunk directory in the log exporter repository 
• Vector sink documentation for Splunk 

Webhook endpoint

To export your logs to a webhook endpoint , create a service using the docker.io/koyeb/log-exporter image with the <VERSION>-webhook tag:

docker.io/koyeb/log-exporter:<VERSION>-webhook

Currently, only Bearer-based authentication is supported.

You can optionally click the Deploy to Koyeb button below to configure the log exporter to ship to a webhook endpoint with much of the configuration pre-filled. Update the environment variables to correspond with your values and choose any other customizations before deploying:

During the deployment, set the following environment variables to configure forwarding to the webhook:

• KOYEB_SERVICE: The name or ID of the Koyeb Service to export logs for. This takes the form of either <APPNAME>/<SERVICE_NAME> or <SERVICE_ID>.
• KOYEB_TOKEN: A Koyeb access token generated in the API section of your Koyeb organization settings .
• ENCODING_CODEC: The message encoding to use. Defaults to json. Supports all Vector codec values  except for avro and csv.
• WEBHOOK_URI: The webhook URI for accepting logs. This should include the protocol, host, and any necessary port and path information.
• WEBHOOK_TOKEN: The access token to use to authenticate to the webhook endpoint.

Additional resources:

• Webhook image tags on Docker Hub 
• Webhook directory in the log exporter repository 
• Vector sink documentation for HTTP server collection 

Inline string

One way to pass a Vector sink configuration is to set an environment variable to a Vector configuration file reformatted as an inline string.

You can optionally click the Deploy to Koyeb button below to configure the log exporter to ship to a custom location using an inline Vector configuration. Much of the Koyeb deployment configuration is pre-filled. Update the environment variables to correspond with your values and choose any other customizations before deploying:

To configure the deployment, you should replace the newline characters in your file with \n characters.

The environment variable should use the following format:

SINK_<FORMAT>_<FILE_NAME>

The variable begins with SINK_ and has the following components:

• <FORMAT>: Specifies the format of the sink configuration file and can be one of JSON, YAML, or TOML.
• <FILE_NAME>: The name the configuration will be given in the Vector configuration directory, without the extension.

For example, assume that you want to use sink configuration file called sink-http.toml with the following contents:

[sinks.hook]
type = "http"
inputs = ["pipe"]
encoding.codec = "json"
uri = "https://my.endpoint.org/logs"
method = "post"

You could use a tool like awk to replace the newline characters with \n by typing:

awk '{printf "%s\\n",$0} END {print ""}' sink-http.toml

A similar alternative using sed (compatible with both Linux and macOS versions) would be:

sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/\\n/g' sink-http.toml

You would then assign the string to the variable SINK_TOML_SINK-HTTP:

SINK_TOML_SINK-HTTP='[sinks.hook]\ntype = "http"\ninputs = ["pipe"]\nencoding.codec = "json"\nuri = "https://my.endpoint.org/logs"\nmethod = "post"\n'

The log exporter image will reconstruct the file from the environment variable and place it in the Vector configuration directory.

GitHub gist

You can use an environment variable to point to a GitHub gist  containing one or more Vector configuration files. The files will all be downloaded and placed within the Vector configuration directory.

You can optionally click the Deploy to Koyeb button below to configure the log exporter to ship to a custom location using a GitHub Gist Vector configuration. Much of the Koyeb deployment configuration is pre-filled. Update the environment variables to correspond with your values and choose any other customizations before deploying:

When configuring the log exporter, set two additional environment variables:

• GIST_<NAME>: A gist to add to the configuration. The <NAME> is an arbitrary label for the gist. The value should be set to the gist ID (the final path component of the gist URL).
• GH_TOKEN: A GitHub (classic) personal access token  with gist permissions. This is only required for private gists.

The filenames will match those specified with the gist itself.

For example, here is a gist containing a Vector configuration file . It defines the following file:

[sinks.hook]
type = "http"
inputs = ["pipe"]
encoding.codec = "json"
uri = "https://webhooks-leok-hash.koyeb.app/webhook"
auth.strategy = "bearer"
auth.token = "7a2dae9e-e94d-4bbf-aad8-733eb05d6d89"

To include the file, you could set a GIST_<NAME> variable to the gist’s ID (60f8c5bec7bbbf6d2cab15cc2bf38def taken from the final part of the gist URL shared above) like this:

GIST_VECTOR_CONFIG=60f8c5bec7bbbf6d2cab15cc2bf38def

The VECTOR_CONFIG portion of the environment variable name is not used when processing the variable, so it is just a descriptive label to help you remember what the gist contains. Since the filename in the gist is called test-webhook-from-gist.toml, this filename will be used in Vector’s configuration directory.

This process works exactly the same for gists that that contain multiple file definitions. Each file will be downloaded and placed in the configuration directory.

Remote file URL

You can include Vector configuration files by providing public URLs where they can be downloaded from. You can set multiple environment variables using this format.

You can optionally click the Deploy to Koyeb button below to configure the log exporter to ship to a custom location using a Vector configuration at a public URL. Much of the Koyeb deployment configuration is pre-filled. Update the environment variables to correspond with your values and choose any other customizations before deploying:

To download a remote hosted Vector configuration file, set environment environment variables with the REMOTE_FILE_ prefix:

• REMOTE_FILE_<NAME>: A URL where a Vector configuration file is hosted. The <NAME> is an arbitrary label for the file. The configuration filename must be the final path component of the provided URL.

For example, you could set an environment variable like this:

REMOTE_FILE_SINK_CONFIG=https://example.com/http-sink.toml

This would create a new file called http-sink.toml with the contents found at the given URL. The SINK_CONFIG part of the environment variable name is not used when processing the variable, so it is just a descriptive label to help you remember what the file represents.