How to get MongoDB Atlas Prometheus Metrics into Observe

Data Ingestion
,
1

This guide walks through how to integrate MongoDB Atlas Prometheus metrics directly into Observe, using the Observe Agent’s Prometheus receiver.

:key: Prerequisites

Before beginning, make sure:

  • You have access to your MongoDB Atlas Project and its organization settings.
  • You have Prometheus integration enabled for your Atlas project.
  • You can modify your Kubernetes cluster’s configuration where the Observe Agent is deployed.

:brick: Step 1. Add Kubernetes Cluster Egress IPs to MongoDB Atlas Allowlist

MongoDB Atlas restricts Prometheus scraping to authorized IPs.
Add your Kubernetes cluster’s egress IP addresses to the Atlas allowlist:

:backhand_index_pointing_right: Follow MongoDB’s documentation:
Configure IP Access List Entries — Atlas — MongoDB Docs

Make sure to add all egress IPs used by your Observe Agent’s namespace or node pool.

:gear: Step 2. Enable Prometheus Integration in MongoDB Atlas

Enable and configure the Prometheus integration in the Atlas UI:

  1. Navigate to Project Settings → Monitoring → Prometheus.
  2. Enable the integration and generate credentials.
  3. Note the username, password, and Prometheus discovery URL provided.

Refer to MongoDB’s official documentation for detailed steps:
Integrate with Prometheus — Atlas — MongoDB Docs

:locked_with_key: Step 3. Create a Kubernetes Secret for MongoDB Credentials

Store your Atlas Prometheus credentials securely in the Observe namespace as a Kubernetes secret:

kubectl create secret generic customer-mongodb-credentials \
  --from-literal=USERNAME=INSERT_USERNAME_HERE \
  --from-literal=PASSWORD=INSERT_PASSWORD_HERE \
  -n observe

Replace INSERT_USERNAME_HERE and INSERT_PASSWORD_HERE with your Atlas credentials.

:rocket: Step 4. Configure the Observe Agent for MongoDB Metrics

Prepare a mongodb.yaml file (values.yaml) with the following configuration, updating the placeholders with your credentials and project ID.

agent:
  config:
    prometheusScraper:
      processors:
        attributes/debug_source_mongodb_prometheus:
          actions:
          - action: insert
            key: debug_source
            value: mongodb_atlas_prometheus
        batch:
          send_batch_max_size: 20480
          send_batch_size: 16384
          timeout: 1s
        memory_limiter:
          check_interval: 5s
          limit_percentage: 80
          spike_limit_percentage: 15
      receivers:
        prometheus/mongodbatlas:
          config:
            scrape_configs:
            - basic_auth:
                username: ${env:CUSTOMER_MONGODB_ATLAS_USERNAME}
                password: ${env:CUSTOMER_MONGODB_ATLAS_PASSWORD}
              http_sd_configs:
              - url: https://cloud.mongodb.com/prometheus/v1.0/groups/INSERT_GROUP_ID_HERE/discovery
                refresh_interval: 347s
                basic_auth:
                  username: ${env:CUSTOMER_MONGODB_ATLAS_USERNAME}
                  password: ${env:CUSTOMER_MONGODB_ATLAS_PASSWORD}
              job_name: customer-mongo-metrics
              metrics_path: /metrics
              scheme: https
              scrape_interval: 300s
      service:
        pipelines:
          metrics/mongodb_atlas_metrics:
            receivers:
            - prometheus/mongodbatlas
            processors:
            - memory_limiter
            - batch
            - attributes/debug_source_mongodb_prometheus
            exporters:
            - prometheusremotewrite/observe

:counterclockwise_arrows_button: Step 5. Upgrade the Observe Agent Helm Chart

Apply the updated configuration to your Observe Agent using Helm:

helm upgrade observe-agent observe/agent -n observe -f mongodb.yaml

This will deploy the new Prometheus receiver for MongoDB Atlas metrics.

:bar_chart: Step 6. Create a Dedicated Dataset for MongoDB Metrics

In Observe:

  1. Navigate to Datasets → Create Dataset.
  2. Use the prometheus dataset as your base.
  3. Filter by debug_source = "mongodb_atlas_prometheus" to isolate MongoDB Atlas metrics.

Name this dataset something like MongoDB Atlas Metrics.