Centralized Logging To Elasticsearch, Logstash, Kibana ELK Stack | Docker Swarm Advanced

How to Set Up Centralized Logging in Docker Swarm Using ELK Stack

Introduction to ELK Stack

• The video introduces the concept of setting up centralized logging in Docker Swarm using the ELK stack, which consists of Elasticsearch, Logstash, and Kibana.

• Filebeat is used to ship logs from Service 1, which writes logs to a file. The logs are shared through a volume for processing.

Differences Between Logging Methods

• Service 2 writes logs to standard output or error; these can be accessed using docker logs or docker service logs.

• A demonstration highlights the difference between logging to a file (Service 1) and logging to standard output (Service 2).

Demonstration of Logging with Nginx Container

• An Nginx container is run with specific parameters: it will automatically remove itself when stopped (--rm) and publish port 81.

• Logs appear in the console as expected when accessing localhost on port 81; they can also be checked using docker logs.

Exploring File-Based Logging

• For Service 2, an Nginx container that writes logs to a file is demonstrated. Accessing standard output does not show any logs.

• To view the access log, one must enter the container and navigate to the log directory.

Setting Up Services for Centralized Logging

• The presenter mentions having already set up Elasticsearch, Logstash, and Kibana (the ELK stack).

• Viewers are directed to follow previous tutorials if they need help setting up ELK.

Preparing Docker Swarm Environment

• The presenter prepares their Docker Swarm environment using Oracle VirtualBox with two VMs: one manager node and one worker node.

• SSH access is established on the registry node to ensure it's operational before proceeding.

Deploying Services Using Compose File

• A Docker Compose file version 3.8 is introduced for deploying services within Docker Swarm.

• Key components of the Compose file include volumes and logging configurations necessary for both services being deployed.

Firebit Configuration and Deployment in Docker

Setting Up Firebit with Engine X

• The speaker discusses sharing the Firelock Engine X to a volume, allowing Firebit to access and read logs from the specified directory (/fpp/log), which will then be shipped to Logstash.

• A configuration file for Firebit is created externally, defined in fivebit.yml, while an internal volume is automatically generated upon deployment of these YAML files. The volume type is set as local.

Deployment Considerations

• If the service runs on different nodes (manager or registry), the corresponding volumes are created on those nodes. Deploying Firebit in global mode ensures it runs across all available nodes.

• In Kubernetes, deploying five replicas of Firebit across five nodes is referred to as a DaemonSet. Access permissions for the configuration must restrict access to only the owner (Firebit user).

Running and Testing Containers

• The speaker demonstrates running a temporary container with specific parameters that ensure automatic removal when no longer needed. Interactive parameters allow entering the container using its ID or name.

• Confirmation of UID and GID values (1000 each) for Firebit users is shown, explaining how these values were derived.

Configuring Logs and Services

• The input directory for logs is defined in fivebit.yml, directing logs from /fpp/log to Logstash at a specified IP address.

• The service deployment command utilizes docker stack deploy, specifying a compose file (dockercompose.yml) that serves merely as a filename rather than indicating use of Docker Compose.

Service Management and Monitoring

• When deploying services, naming conventions prepend stack names to service names (e.g., someover_nginx1). This applies similarly to volumes created during deployment.

• After confirming successful deployment globally, two replicas of Firebit are noted alongside other services like Nginx. Accessing services generates logs intended for ELK stack storage.

Verifying Log Shipping

• The speaker accesses ports 82 and 81 via browser using manager node IP addresses; services produce logs expectedly sent to Kibana for monitoring.

Centralized Logging Setup in Docker Swarm Using ELK Stack

Initial Setup and Index Pattern

• The username is set to "elastic" and the password is verified. An index pattern has already been created, allowing direct access to logs through the discovery menu.

• Logs are being sourced from a file, indicating that the index pattern may not be functioning as expected.

Switching to Calf Driver Logs

• Transitioning to view logs from the Calf driver, confirming successful setup of centralized logging within Docker Swarm using the ELK stack.

• Additional tips are provided on creating containers that write logs either to a file or standard output.

Working with Nginx Container

• Entering the Nginx content area within an official Nginx image container, which writes logs to standard output.

• Clearing the screen before demonstrating how Nginx typically writes logs to its log directory.

Redirecting Logs for Standard Output

• To redirect logs to standard output, they must be directed to /dev/stdout or /dev/stderr.

• The contents of the log directory show access and error logs redirected appropriately.

Differences Between Official and Customized Images

• A customized image was modified by removing redirection; thus, it now writes logs directly to a log file instead of standard output.

• Comparison made between official images (which redirect logs correctly) and customized images (which do not).

Conclusion and Call-to-action

• The presenter expresses gratitude for viewers' patience with their English skills while sharing knowledge about logging setups.