How to set up Jenkins integration via Observe Agent

[arthurdayton]

This guide demonstrates how to set up Jenkins integrations to Observe using Observe Agent and Jenkins’ OTEL Plugin

This guide is meant to be a replacement for Jenkins CI/CD integration - Observability Cloud documentation which is now deprecated. It serves as “Jenkins 2.0” without the need for an app and instead provides logs and metrics datasets out of the box.

As a result, the integrations helps you monitor the activity and health of your Jenkins pipeline with logs & metrics datasets that include information about your Jenkins jobs, builds, build logs, nodes, agents, and more.

Instructions

Requirements:

• Jenkins Instance

• Amazon Linux 2+ EC2 (can also use Ubuntu)

• An Observe Tenant

If you have fluentbit running, ensure you have that disabled on the machine before installing observe-agent due to port clashing

If you have the stats gatherer plugin in Jenkins, disable that as well

Observe-Agent Setup

1. On your Observe tenant, navigate to “Add Data” and click “Linux” under “Observe Agent” (this may need to enabled as part of a feature flag for your tenant if it doesn’t show up)

2. Follow the instructions on the Observe-Agent installation page. Namely, you will need to:

   1. Create an ingest token

   2. Select the data to send to Observe (leave default to logs & metrics)

   3. Install Observe-Agent on your EC2

   4. Configure the Observe Agent

      1. Ensure that you have created a token before running this command!

   5. Start the Observe-Agent and ensure that it’s running successfully

   6. Verify you’re receiving data from Observe Agent

Check that ${YOUR_INGEST_TOKEN} is pre-populated with the actual ingest token before running this command

3. Running observe-agent status should show something similar to the following with “Status: Running”

   If there are any issues configuring observe-agent itself, follow the docs here: Troubleshooting - Observability Cloud documentation

   

4. Edit the observe-agent.yaml to include jenkins log.

• Run sudo vi /etc/observe-agent/observe-agent.yaml

#Under host_monitoring, add jenkins:
host_monitoring:
    enabled: true
    logs:
        enabled: true
        include:
            - /var/log/**/*.log
            - /var/log/syslog
            - /var/lib/jenkins/**/*.log # system logs
            - /var/lib/jenkins/jobs/**/log   #  build logs

• Run sudo systemctl restart observe-agent
• Run observe-agent status to ensure Observe Agent is still running after the restart.

Jenkins Setup

1. Ensure you have a Jenkins instance running on your machine

2. Navigate to “Plugins” from the main home page

   

3. Install the OpenTelemetry Plugin and ensure it’s enabled

   

4. Once installed, navigate to “System” from the main home page to configure the plugin

   

5. Under OpenTelemetry, set the OTLP Endpoint to point to the endpoint provided by the Observe-agent

   1. This will be http://localhost:4318

   2. Set “No Authentication”

      

If installing on Kubernetes, the endpoint will be the following:

http://observe-agent-forwarder.observe.svc.cluster.local:4318

Install Observe Agent on Kubernetes - Observability Cloud documentation

6. Under “Advanced”, ensure your configuration properties are set as below

   

7. Click “Save” and “Apply”

8. Navigate back to the home page and click “System Log”

   

9. Ensure there are no errors such as the following example in the logs

   

Jenkins Data in Observe

1. Once both Observe Agent and Jenkins are configured properly, data should appear in Observe

2. First ensure data is coming from observe-agent to Observe

   1. Under Overview, check that health is “Healthy” for data

      

3. Navigate to OpenTelemetry Logs in Log Explorer to view Jenkins Logs

   1. Extract attributes['log.file.path']) and filter for anything with jenkins to find Jenkins logs that are written to disk

   2. Namely:

      • /var/lib/jenkins/**/*.log # system logs

      • /var/lib/jenkins/jobs/**/log # build logs

        

4. Navigate to OpenTelemetry Metrics in Metrics Explorer to find your Jenkins metrics

   

5. (Optional) If tracing is configured for your tenant, you may also receive trace data from Jenkins as part of the same plugin configuration. All other attributes are shown here