How should environment variables and configuration files be used in ML containers?

An ML container should get deployment-specific settings when it starts, instead of requiring a new image for every environment. Use environment variables for small, simple settings and mounted configuration files for larger structured settings. Keep secrets in an appropriate secret-management system.

This builds on configuration-as-code: the focus here is how configuration reaches a running training or inference container.

Beginner-friendly example

A food-delivery company serves a model that predicts delivery time. The same tested container image runs in staging and production, but each environment uses a different approved model version, model-storage location, and prediction timeout.

Instead of rebuilding the image, the team supplies the settings at startup.

1. Use environment variables for small values

The Dockerfile can provide harmless defaults:

DOCKERFILE
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY app/ ./app/
ENV PREDICTION_TIMEOUT_SECONDS=2
CMD ["python", "-m", "app.main"]

The application reads the settings and validates them:

PYTHON
import os

model_version = os.environ["MODEL_VERSION"]
model_uri = os.environ["MODEL_URI"]
timeout = float(os.getenv("PREDICTION_TIMEOUT_SECONDS", "2"))

if timeout <= 0:
    raise ValueError("Prediction timeout must be positive")

Required settings such as MODEL_VERSION and MODEL_URI should fail clearly if missing. A reasonable default may be suitable for a non-sensitive timeout but not for a production model identity.

Run the same image with a non-secret environment file:

# staging.env
MODEL_VERSION=staging-v7
MODEL_URI=s3://example-ml-artifacts/staging/model-v7
PREDICTION_TIMEOUT_SECONDS=3

BASH
docker run --rm --env-file staging.env delivery-api:1.0

--env-file injects the variables when Docker starts the container. The shown storage address is illustrative; the application still needs authorized access and code that loads the model.

2. Use configuration files for structured settings

Some ML applications have a group of related options—such as preprocessing choices, input feature schema, batch size, and postprocessing thresholds—that are easier to review as YAML or JSON.

For example, inference.yaml might contain:

YAML
features:
  distance_unit: km
  missing_preparation_time: median
inference:
  batch_size: 16
  confidence_threshold: 0.75

Mount this non-secret file instead of embedding environment-specific settings in the container:

BASH
docker run --rm \
  --env MODEL_VERSION=staging-v7 \
  --mount type=bind,src="$(pwd)/inference.yaml",dst=/etc/ml/inference.yaml,readonly \
  --env ML_CONFIG_PATH=/etc/ml/inference.yaml \
  delivery-api:1.0

The application must actually parse ML_CONFIG_PATH, and the host file must exist. A read-only mount keeps the service from changing the configuration file accidentally.

Rule of thumb: Environment variables work well for simple scalar values and file locations; mounted files work well for structured configuration. Use both when helpful.

3. Keep sensitive values separate

Do not hard-code API tokens, private keys, or database passwords in `ENV`, `ARG`, Dockerfiles, Git-tracked YAML, or committed `.env` files.

Runtime environment variables can sometimes be exposed through process inspection, diagnostic dumps, logs, or container-management interfaces. Prefer a supported secret manager or secret mount, with scoped access, and avoid logging secret values.

For cloud storage, prefer workload identities and short-lived credentials when available rather than copying long-lived keys into an image.

How does this work in Kubernetes?

• A ConfigMap can provide non-confidential settings as environment variables or files mounted into a Pod.
• A Secret is a different Kubernetes object for sensitive values. It still needs appropriate RBAC, encryption-at-rest configuration where applicable, and careful access.
• A deployment can point the service at an approved model version and configuration without rebuilding its image.

Watch out for updates:

• Environment variables derived from a ConfigMap do not automatically change inside an already-running process when the ConfigMap changes; restart or roll out new Pods to pick up the new values.
• Mounted ConfigMap files may be updated eventually, but the application must reread them or be restarted. A subPath mount does not receive normal ConfigMap updates.
• For reproducible model releases, pin or version the effective configuration and record it with the model and container image identifiers. Do not silently change preprocessing behavior while a model continues serving.

Avoid these common mistakes

• Building separate images for staging and production just to change a timeout or model URI.
• Treating `EXPOSE`, `ENV`, or a mounted file as automatic application logic: the process must explicitly read and validate configuration.
• Copying local `.env` files into the image: exclude local settings and secrets using .dockerignore.
• Overriding settings in too many places: document precedence between Dockerfile defaults, docker run, Compose, and deployment settings.
• Mixing settings with the model itself: track the configuration version used to evaluate and deploy the model.

Earlier questions covered configuration-as-code, environment parity, and secure credentials in general. This lesson explains the specific container delivery mechanisms, including environment files, read-only mounts, ConfigMaps, and restart behavior.

In short: Build the ML container once; inject non-secret settings at runtime through environment variables or configuration files, deliver sensitive values securely, validate the effective configuration, and record it alongside the released model.