How do I update Observe K8 Agent resourcedetection/cloud detectors?

Data Ingestion
, , ,
1

This guide applies to the Observe Agent (based on OpenTelemetry Collector) deployments using Helm charts, focusing on updating the resourcedetection processor to include desired detectors and exclude the eks detector to avoid errors like:

2025-05-14T20:44:52.531Z warn internal/resourcedetection.go:131 failed to detect resource {"kind": "processor", "name": "resourcedetection/cloud", "pipeline": "logs/observe-forward", "error": "isEks() error retrieving auth configmap: failed to retrieve ConfigMap kube-system/aws-auth: configmaps \"aws-auth\" not found"}

Understanding OpenTelemetry Resource Detectors:

  1. Resource detectors automatically detect metadata about the environment (e.g., Kubernetes, AWS, GCP, or host details).
  2. Common detectors include:
  • env: Reads environment variables.
  • gcp: Detects Google Cloud metadata.
  • aws: Detects AWS metadata (e.g., EC2, ECS, but excluding EKS).
  • k8s: Detects Kubernetes metadata (e.g., pod, namespace, node).
  • host: Detects host-level attributes (e.g., hostname, OS type).
  1. Observe is currently configured for the following detectors "eks", "gcp", "ecs", "ec2", "azure"
  2. The eks detector, part of the aws detector, requires access to the kube-system/aws-auth ConfigMap, which may not exist in non-EKS or misconfigured environments, causing errors.

1. Retrieve the Current values.yaml File

If you already have your values.yaml file, you can edit it directly and skip to step 2. Otherwise, download the current configuration (replace <YOUR_NAMESPACE> with your actual namespace):

export NAMESPACE=<YOUR_NAMESPACE> helm get values observe-agent -n $NAMESPACE -o yaml > current-values.yaml

2. Create and Edit the Updated values.yaml File

Make a copy of the current configuration and clean it up:

cp current-values.yaml updated-values.yaml sed -i.bak '/^USER-SUPPLIED VALUES:$/d' updated-values.yaml

Open updated-values.yaml in a text editor and locate the agent.config.global section:

agent: config: global:

Add or update the overrides.processors.resourcedetection/cloud section to specify the desired detectors, explicitly excluding eks. For example:

agent: config: global: ... <keep existing values> ... overrides: processors: resourcedetection/cloud: detectors: ["gcp", "ecs", "ec2", "azure"]

This configuration ensures the resourcedetection processor uses only the specified detectors (gcp, ecs, ec2, azure) and excludes eks to prevent errors.

3. Apply the Updated Configuration

Apply the changes using Helm (replace <YOUR_NAMESPACE> with your actual namespace):

helm upgrade observe-agent observe/agent -n $NAMESPACE -f updated-values.yaml

This configuration ensures the resourcedetection processor uses only the specified detectors (gcp, ecs, ec2, azure) and excludes eks to prevent errors.