Skip to main content

sprout_monitoring

Comprehensive monitoring and logging solution using Prometheus, Grafana, Loki, and Alertmanager.

Overview​

The sprout_monitoring repository provides:

  • Prometheus for metrics collection
  • Grafana for visualization and dashboards
  • Loki for centralized logging
  • Alertmanager for alerting

Technology Stack​

  • Prometheus: Metrics collection and storage
  • Grafana: Visualization and dashboards
  • Loki: Centralized logging
  • Alertmanager: Alert routing and notification
  • Pushgateway: Metrics from batch jobs
  • Docker Compose: Service orchestration

Getting Started​

Prerequisites​

  • Docker and Docker Compose
  • Access to monitored services

Setup​

  1. Configure services: Edit config/services.yaml with your service URLs

  2. Generate Prometheus config:

    ./scripts/generate-prometheus-config.sh

  3. Start monitoring stack:

    docker-compose up -d

  4. Access dashboards:

Architecture​

See Architecture Documentation for detailed architecture.

Monitored Services​

HTTP Metrics Services​

  • sprout_backend (Port 3000) - NestJS API
  • sprout_etl (Port 3001) - Fastify ETL service

Pushgateway Services​

  • sprout_backend Worker (Port 4001) - Background worker
  • sprout_etl Worker (Port 4002) - ETL worker

Key Features​

Metrics Collection​

  • HTTP scraping from /metrics endpoints
  • Pushgateway for batch jobs and workers
  • Custom service labeling
  • Automatic service identification

Dashboards​

  • Sprout System Health - Overall system status
  • ETL System Monitoring - ETL pipeline health
  • Inventory Monitoring - Inventory metrics
  • TickOps Quota Monitoring - API quota tracking

Alerting​

  • Service down alerts
  • High error rate alerts
  • Performance degradation alerts
  • Worker health monitoring

Configuration​

Service Configuration​

Edit config/services.yaml:

services:
  etl_fastify:
    host: "host.docker.internal"
    port: 3001
    enabled: true
    
  nestjs_monolith:
    host: "host.docker.internal"
    port: 3000
    enabled: true

Prometheus Configuration​

Generated from config/services.yaml:

./scripts/generate-prometheus-config.sh

Grafana Provisioning​

  • Datasources: grafana/provisioning/datasources/
  • Dashboards: grafana/provisioning/dashboards/
  • Alerting: grafana/provisioning/alerting/

Deployment​

Local Development​

docker-compose up -d

Production​

./scripts/deploy-monitoring-droplet.sh

Scripts​

  • generate-prometheus-config.sh - Generate Prometheus config
  • deploy-monitoring-droplet.sh - Deploy to production
  • setup-monitoring.sh - Setup monitoring stack
  • run-monitoring.sh - Run monitoring services

Documentation​