How to create a Train?

In this documentation, we will show you how to create new trains and how to store them in order to allow the usage of the train in PADME.

How to create new trains

Trains are essentially Docker images. This means a wide range of programming languages and frameworks is supported. To create a new train, you need to create a Dockerfile, describing the build process for the Docker image. A guide on how to create Dockerfiles is provided here. For our purpose mostly FROM, RUN, COPY, LABEL, and CMD will be used.

In this documentation, we will now describe the parts of the Dockerfile that are specific to PADME and the creation of trains.

Specifying Variables

To allow the configuration of your train docker images, we use environment variables. Whenever a train arrives at a station, values for these variables need to be specified by the station software user. This feature can for example be used to configure connection strings, names of datasets, credentials, etc.

All environment variables supported by the train need to be set via a LABEL called ‘envs’. You can see an example of this label in the Dockerfile at the end of this section.

This envs label contains a description of the supported environment variables in form of a JSON array. For each environment variable, the following needs to be provided:

{
  "name": "the name of the variable",
  "type": "number|password|text|url|select",
  "required": true,
  "options": [
    "only needed",
    "when using select"
  ]
}

The property ‘name’ is the name that gets displayed to the station software user. Please be aware that since the name will be used as an environment variable, it cannot contain any spaces. Besides the name, we also support five different types: number, password, text, url, and select. Depending on the type, a different visualization is used in the station software. For example, a password will not be visible in plain text. When using the ‘select’ type, you can provide the selectable options via the ‘options’ array. Moreover, it is possible to mark variables as required.

The following image shows an example of how these environment variables will be visualized to a station software user. This example shows the train build by the Dockerfile at the end of this section.

Since the JSON array containing the environment variable descriptions is stored in the Dockerfile as a string, the quotes contained in the array itself need to be escaped. This can be achieved by putting a backslash before every quote. For example “name” would be replaced with \”name\”. This can also be done with many online tools or sed:

echo 'envs=[...]' | sed 's/"/\\"/g'

The following code is an example of a train image using python. The specified environment variables are the ones shown in the picture above.

Dockerfile

FROM python:3

WORKDIR /usr/src/app/

#Required env vars
LABEL envs="[{\"name\":\"ADDRESS\",\"type\":\"url\",\"required\":true}, \
{\"name\":\"PORT\",\"type\":\"number\",\"required\":true}, \
{\"name\":\"ALGORITHM_TYPE\",\"type\":\"select\",\"options\":[\"linear\",\"loop\"],\"required\":true},\
{\"name\":\"PASSWORD\",\"type\":\"password\",\"required\":true}, \
{\"name\":\"MESSAGE\",\"type\":\"text\",\"required\":false}]"

## Install all required packages
COPY packages.txt .
RUN pip3 install -r packages.txt

## Copy Source
COPY . .

## Run
CMD [ "python", "main.py" ]

Using environment Variables

In the previous section, we discussed how to specify which environment variables are needed for your train. We will now show a small example of how to use these variables.

Since values for the environment variables are specified in the station software when the container for the train is created, we can use the values in the train simply be reading the environment variables during runtime. For example this can be achieved in python by the following code:

address = str(os.environ['ADDRESS'])
port = str(os.environ['PORT'])
algotype = str(os.environ['ALGORITHM_TYPE'])
password = str(os.environ['PASSWORD'])

Where to store trains

All trains are stored in our Train Depot. This Train Depot is a git repository containing all PADME trains. If you don’t have access yet, please feel free to contact us via the information provided in the ‘Find Us’ section on the right.

To make a train available in PADME, you simply need to create a new folder in the train depot, containing your code and the Dockerfile described in the previous section. On every commit to the repository, changed trains will be automatically build and made available in PADME.

The build progress of your train can be monitored via the CI/CD menu in GitLab. Below is an example of a finished build:

If you need to change your train at a later point in time, simply change the code and docker image, and everything will be rebuild and updated automatically.

Now you are all set for creating trains in PADME. If you have further questions please do not hesitate to contact us.