Skip to main content

Deployment Guide

Local Development

Step-by-step local setup

Local cluster demo

Production

  • Docker Compose production manifest
  • Kubernetes manifests and scaling guidance

Production checklist

Example: Docker Compose (production)

Below is a minimal production-ready Docker Compose file. Adjust resource limits, secrets, and storage for your environment.

Advanced Kubernetes deployment

For high-availability and scalability, use Kubernetes. Example manifests: Master Deployment:
Worker DaemonSet:
Secrets and Config:
Monitoring and scaling:
  • Use Prometheus and Grafana for metrics. Scrape /metrics endpoint.
  • Use Jaeger for distributed tracing. Export traces from all services.
  • Use HorizontalPodAutoscaler (HPA) for worker pools:

Advanced Troubleshooting

  • Check container logs for errors on startup and shutdown.
  • Use kubectl describe and kubectl logs for debugging pods.
  • Validate DB and Redis connectivity from within pods.
  • Use loomos jobs list and /metrics endpoints to monitor job and cluster health.
  • Confirm all services are healthy via /healthz endpoints.
  • Enable debug logging for more verbose output.
  • Set up alerting for job failures, resource exhaustion, and unhealthy nodes.

Best Practices & Security

  • Always use TLS for all endpoints in production.
  • Store secrets in Kubernetes secrets or a vault, never in plain text.
  • Use resource limits and requests for all containers.
  • Enable audit logging and event tracing for compliance.
  • Regularly back up Postgres and Redis data volumes.
  • Use S3-compatible storage for model artifacts and logs.
  • Test failover and recovery procedures regularly. See docker-compose.prod.yml in the repository for a full production example.

Configuration

Environment variables and monitoring endpoints