OpenTelemetry in Node.js - Traces, Metrics and Logs

Introduction to OpenTelemetry

Overview of OpenTelemetry

• OpenTelemetry is an observability framework that enhances visibility into application behavior.

• The speaker encourages viewers to check out a related video titled "What is OpenTelemetry" for theoretical insights.

Setting Up OpenTelemetry in Node.js

Initial Setup

• The implementation will be demonstrated using a basic Express application, which includes a simple endpoint for rolling dice.

• The application returns an array of dice rolls based on the number specified in the query parameter.

Application Testing

• The speaker tests the endpoint using Insomnia, confirming it returns the expected results.

Complexity and Purpose of Example Application

Rationale Behind Complexity

• The example's complexity serves to illustrate how OpenTelemetry can scale with more intricate applications.

• It allows demonstration of nested spans, context propagation, and log correlation.

Installing Required Packages

Package Installation

• Necessary packages include the Node SDK for OpenTelemetry, API, auto-instrumentation node package, SDK for metrics and traces, resources, and semantic conventions.

Creating Instrumentation File

Instrumentation Setup

• A file named instrumentation.js is created to initialize OpenTelemetry with default configuration options.

Configuration Options

• The trace exporter is set up to use console logging for debugging purposes before switching to production exporters later.

Adding Metrics Support

Metric Reader Configuration

• A periodic exporting metric reader is configured alongside a console metric exporter to collect metrics at user-defined intervals.

Auto Instrumentation Library

Benefits of Auto Instrumentation

• This library automatically creates traces and metrics for popular libraries and frameworks like AWS Lambda and Express.

Implementation Details

• It simplifies tracking HTTP requests by generating traces without manual setup while allowing custom metrics creation as needed.

Defining Resources

Resource Definition

• Resources are defined using semantic conventions that help standardize communication across services.

Attributes Used

• Service name: "dice roll app"

• Version: "1.0"

Starting the SDK

Running the Application

• It's crucial to run the instrumentation file before starting the main application code to capture its entire lifecycle.

Execution Command

• Use pmpm DLX or node --import commands depending on module type (ESM or CommonJS).

Observing Traces from Requests

Trace Observation

• After sending requests via Insomnia, trace data appears in logs due to auto instrumentation setup.

Trace Data Insights

• Includes resource attributes such as service name/version and request details like URL and method status (200).

Creating Custom Traces

Acquiring a Tracer

• To create custom traces, acquire a tracer using getTracer() function from Open Telemetry API with scope name parameters.

Span Creation

• Spans represent units of work; utilize startActiveSpan() method for better context management during operations.

Enhancing Trace Information

Nested Spans Implementation

• Implement nested spans by passing additional parameters (like index values), improving trace clarity through parent-child relationships between functions.

Adding Attributes & Events

• Attach key-value pairs as attributes within spans for detailed tracking information about operations performed during execution.

Utilizing Span Events

Event Logging within Spans

• Add human-readable events within spans using addEvent(), providing contextual information about specific actions taken during execution.

Handling Statuses & Exceptions

Status Management in Spans

• Set span statuses indicating success or failure; use predefined codes from Open Telemetry API (e.g., error).

Exception Recording Strategy

• Record exceptions within spans using try-catch blocks; this aids in debugging by capturing error messages directly associated with specific operations.

Setting Up Metrics in OpenTelemetry

Introduction to Metrics

• The process of setting up metrics is similar to traces, requiring the acquisition of a meter using getMeter.

• A constant named meter is created by calling getMeter, passing the instrumentation scope as an argument.

• It’s recommended to use getMeter in specific application areas rather than exporting it globally.

Creating Counters

• Counters are introduced as a simple metric tool for measuring non-negative increasing values.

• An example counter, di-li.rolls.counter, is created to count how often dice have been rolled, utilizing the meter's create counter method.

• The counter can only increase; however, an up-down counter can be used if needed.

Implementing Histograms

• Histograms measure value distributions over time and can be set up manually or through auto-instrumentation.

• A histogram named HTTP.server.duration is created with options like description and unit (milliseconds).

• Execution time is recorded by capturing start and end times around function execution.

Running Metrics and Observations

Testing Metrics Output

• After running the code with newly created metrics, results should appear in the console after some time due to periodic metric exporting.

• Custom metrics such as counters and histograms are displayed alongside auto-instrumented metrics.

Logging with OpenTelemetry

Integrating Logging Providers

• The logs API for OpenTelemetry JavaScript is under development but expected to remain stable for current implementations.

• Auto instrumentation includes support for popular logging providers like Winston, making production-grade logging easier.

Setting Up Winston Logger

• Necessary packages including Winston and its transport package are installed for logging integration.

• Log record processors are configured using a simple log record processor from the SDK of logs.

Configuring Exporters for Observability

Sending Data to OpenTelemetry Collector

• To send data off to an observability tool, exporters need configuration changes in the instrumentation file.

• Installation commands for trace, metric, and log exporters using OpenTelemetry Protocol (OTP).

Setting Up Better Stack Integration

• Instructions provided on creating a source in Better Stack for connecting telemetry data from OpenTelemetry collector.

Visualizing Logs and Metrics

Viewing Data on Better Stack Dashboard

• Once requests are sent through endpoints, logs and metrics appear on Better Stack's dashboard with detailed views available.

Creating Custom Charts

• Users can create custom charts based on their own metrics within Better Stack's interface.