How to add new datasets to Log Explorer in Observe

Log Management
, ,
1

Overview

In the Observe platform, logs contained in a generic dataset do not automatically appear in the Log Explorer. To make them visible, you must configure the dataset to be recognized as containing logs. This is achieved by editing the dataset and using the “log” option of the interface OPAL command. Once configured, the logs will be available for visualization in the Log Explorer.

Steps to Add a Dataset to Log Explorer

To enable a dataset to appear in the Log Explorer, follow these steps:

  1. Edit the Dataset: Access the dataset in the Observe platform that you want to configure for Log Explorer.
  2. Apply the Interface OPAL Command: Use the interface verb to map fields to a log interface. The OPAL code required is as follows:
// Make a string column named "log" from the "FIELDS.log_msg" object so interface models it
make_col log:string(FIELDS.log_msg)

// Declare the "log" interface. By default, this will extract the log message 
// from the FIELDS object.
interface "log"

// Optionally, you can specify a different column to use as the log message. 
// For example:
// interface "log", log:<messageColumn> 
// where <messageColumn> is the name of the column you want to use instead.

Explanation of the OPAL Code

  • make_col log:string(FIELDS.log_msg): This creates a string column named log by extracting the log message from the FIELDS.log_msg object in the dataset.
  • interface “log”: This informs Observe that the dataset contains logs, enabling it to be visualized in the Log Explorer.

Conditions Set by the Log Interface

Applying the log interface to a dataset ensures the following:

  • Each row in the dataset represents an event in a time series.
  • A field named log contains the message from the application.

The data itself remains unchanged, but these conditions allow the Log Explorer to interpret and display the dataset as logs.

Additional Notes

  • Ensure the column mapped to log is of type string. If it is not, use OPAL to convert it to a string before applying the interface “log” command.
  • For more details on the interface verb and its usage, refer to the OPAL documentation in the Observe platform.

By following these steps, your dataset will be properly configured to appear in the Log Explorer, enabling seamless log visualization and analysis.

5

Specifying different log columns is particularly valuable and might be worthwhile calling out here. e.g.

interface "log", log:message
6

@andy Thank you for the helpful suggestion. I’ve updated the post to reflect your feedback.