Skip to main content

Quick Setup Guide - Monitoring Stack with Custom Domain

This guide will help you set up the monitoring stack on your DigitalOcean droplet with your custom domain.

Prerequisites​

  • ✅ DigitalOcean droplet with Ubuntu 22.04 LTS
  • ✅ Domain managed by Cloudflare
  • ✅ SSH access to your droplet
  • ✅ Monitoring stack repository cloned on the droplet

Step 1: SSH to Your Droplet​

ssh root@your_droplet_ip

Step 2: Navigate to Monitoring Stack Directory​

cd /path/to/monitoring-stack

Step 3: Set Your Domain Variables​

# Set your domain (replace with your actual domain)
export YOUR_DOMAIN=yourdomain.com

# Optional: Customize subdomains (defaults shown)
export GRAFANA_SUBDOMAIN=monitoring
export PROMETHEUS_SUBDOMAIN=prometheus
export ALERTMANAGER_SUBDOMAIN=alertmanager

Step 4: Run the Setup Script​

./scripts/setup-custom-domain-tunnel.sh

This script will:

  • Install Cloudflare tunnel (cloudflared)
  • Create a tunnel for your domain
  • Set up DNS records automatically
  • Start the monitoring stack (Prometheus, Grafana, Alertmanager)
  • Configure systemd service for automatic startup

Step 5: Access Your Monitoring Stack​

After the script completes, your monitoring stack will be accessible at:

  • Grafana Dashboard: https://monitoring.yourdomain.com
  • Prometheus: https://prometheus.yourdomain.com
  • Alertmanager: https://alertmanager.yourdomain.com

Step 6: Initial Configuration​

1. Change Grafana Password​

  • Go to https://monitoring.yourdomain.com
  • Login with admin/admin
  • Go to Settings → Users → Change Password

2. Update Service Configuration​

Edit config/services.yaml to point to your actual services:

services:
  etl_fastify:
    host: "your-etl-server-ip"  # Replace with actual IP
    port: 3001
    enabled: true
    
  nestjs_monolith:
    host: "your-nestjs-server-ip"  # Replace with actual IP
    port: 3000
    enabled: true

3. Regenerate Prometheus Configuration​

./scripts/generate-prometheus-config.sh
docker-compose restart prometheus

4. Configure Alerting (Optional)​

Note: All alerting is now managed in Grafana. Prometheus no longer loads alert rules from etl-alerts.yml. Edit grafana/provisioning/alerting/alertmanager.yml to add your Slack webhook or email notifications.

Management Commands​

# View all logs
docker-compose logs -f

# Restart the stack
docker-compose restart

# Stop the stack
docker-compose down

# Check tunnel status
cloudflared tunnel info sprout-monitoring

# View tunnel logs
journalctl -u cloudflared -f

Troubleshooting​

Services Not Appearing​

  • Check if your services are running and accessible
  • Verify the IP addresses in config/services.yaml
  • Check firewall settings on your service servers

Tunnel Not Working​

  • Check tunnel status: cloudflared tunnel info sprout-monitoring
  • View logs: journalctl -u cloudflared -f
  • Verify DNS records in Cloudflare dashboard

Dashboard Not Loading​

  • Check if Grafana is running: docker ps
  • View Grafana logs: docker-compose logs grafana
  • Verify Prometheus data source in Grafana

Security Notes​

  • ✅ No ports need to be opened on your droplet
  • ✅ All traffic is encrypted through Cloudflare tunnel
  • ✅ SSL certificates are automatically managed
  • ✅ Access is controlled through your domain

Next Steps​

  1. Configure Dashboards: Import or create custom dashboards in Grafana
  2. Set Up Alerts: Configure alerting rules for your services
  3. Monitor Resources: Set up monitoring for your droplet's system resources
  4. Backup Configuration: Regularly backup your configuration files

Support​

If you encounter issues:

  1. Check the logs: docker-compose logs -f
  2. Verify tunnel status: cloudflared tunnel info sprout-monitoring
  3. Test connectivity to your services
  4. Review the main README.md for detailed troubleshooting