Splunk SIEM Explained:A Beginner's Guide In Arabic

Introduction to SIM and Splunk

Opening Remarks

• May God keep you well, dear engineers. The session begins with a prayer for knowledge and guidance.

• The speaker introduces the topic of the playlist: SimSolution Splunk, focusing on how to navigate from beginner to advanced levels in using Splunk.

Understanding SIM and Its Components

• The term "SIM" stands for Security Information Management, which involves four key components: Collection, Normalization, Analysis, and Alerting.

• Collection refers to gathering logs from various sources such as networks or hosts (e.g., workstations, servers). This is essential for effective security management.

Log Sources and Their Importance

• Logs can originate from different devices like switches, routers, Windows/Linux workstations, and web servers; understanding these sources is crucial for log collection strategies.

• Instead of manually downloading logs from each endpoint, a centralized solution like SIM collects all logs efficiently in one place. This streamlines the process significantly.

Stages of Log Processing

Collecting Logs

• The first stage in the SIM process is collecting logs from various endpoints into a central system where they can be managed effectively.

Normalizing Data

• After collection, normalization occurs where collected data is organized into structured formats (like tables) for easier analysis later on. This step ensures consistency across different log types.

Analyzing Logs

• In the analysis phase, specific features are utilized to examine logs based on predefined rules (or "rolls"). These rolls help identify significant events or anomalies within the data collected.

Alerting Mechanism

Understanding Splunk and Its Components

Introduction to Log Sources

• The discussion begins with the potential sources of logs, such as WAF (Web Application Firewall) or router switches, indicating that various systems can provide log data.

• Emphasis is placed on gathering detailed information from network sources, including user sessions and traffic analysis.

Overview of SIM Tools

• The speaker introduces Splunk as a type of Security Information Management (SIM) solution used for log management and analysis.

• Understanding the components of Splunk is crucial for effective implementation and preparation for technical interviews.

Key Components of Splunk

• Splunk consists of three main components: forwarders, indexers, and search engines. Each plays a distinct role in log collection and analysis.

• Forwarders act as agents installed on endpoints to collect logs without needing manual intervention at each device.

Log Processing Workflow

• After collecting logs, forwarders send them to an indexer where they are processed into tables through parsing filters.

• The final stage involves accessing these organized logs via the search engine for investigation purposes.

Types of Indexers

• There are two types of indexers: light orderers (for simple endpoint logging) and heavy orderers (which offer advanced features like additional analytics).

• Heavy orderers enhance log processing capabilities compared to light orderers by providing more complex filtering options.

Exploring the Features of Splunk

Accessing the Search Engine

• Upon opening Splunk, users interact with its search engine to write queries for data retrieval and analysis.

Applications within Splunk

• Users can enhance their experience by integrating apps that improve functionality; examples include cloud-related applications for data collection.

Data Organization in Splunk

Understanding Data Upload in Splunk

Overview of App Data and Upload Methods

• The concept of "App Data" is introduced, explaining its role in uploading logs or data to Splunk.

• Users can upload data through various methods, including a direct upload option and using a mentor or forwarder.

• For enterprise-level installations, local uploads are not feasible; instead, users must utilize forwarders for data collection from endpoint devices.

Configuring Upload Settings

• Configuration options allow users to connect Splunk to specific ports on endpoint devices for log transmission.

• The upload process begins with selecting the file containing logs, leading to stages that prepare the data for analysis.

Log Detection and Source Type Identification

• Upon uploading logs, Splunk automatically detects the format (e.g., JSON), categorizing it into tables for easier access.

• The source type indicates the format of uploaded logs, which can vary; default detection helps streamline this process.

Detailed Log Information and Indexing

• Important metadata such as event time, source country, and protocol are captured during the upload process.

• The "host field value" represents the machine name from which logs were uploaded; users can customize this information as needed.

Finalizing Upload and Querying Data

• Users have options to specify indexes where uploaded datasets will be stored; common choices include a default index named "main."

Understanding Log Analysis and Filtering Techniques

Introduction to Log Analysis

• The speaker discusses the process of returning logs and introduces a tool called "Configurator" for log analysis.

• Emphasis is placed on an application named "search and report," which is utilized for analyzing logs in 99% of cases.

Data Summary Insights

• The data summary feature indicates the source of logs, such as "Wind 10," helping users identify where the logs originate from.

• A specific example highlights that some logs are related to VPN activity, showcasing how to interpret log sources effectively.

Time Functionality in Log Analysis

• The importance of setting a time range for analysis is discussed, allowing users to focus on specific periods when investigating logs.

• Users can input exact dates and times (e.g., 4/22/20025 at 6:00), ensuring that only relevant logs are displayed during investigations.

Analyzing Logs by User Activity

• The speaker explains how to view the number of uploaded logs and filter them by user names, providing insights into individual user activities.

• A query about a user named Malina reveals that they have 60 associated logs, demonstrating how to extract specific user data from larger datasets.

Advanced Filtering Techniques

• The speaker describes using filters with commands like "status" and "username" to refine search results further.

User Actions and Log Analysis

Overview of User Malena's Logs

• The user, Malena, is from the United States, with a log containing 60 events. This indicates a significant amount of activity recorded in her logs.

• Malena's IP address is identified, along with two actions: "Teardown" and "Bill," marking specific interactions on a given date.

Username Identification

• The username associated with Malena’s IP is revealed to be "Smith," indicating successful identification through log analysis.

Event Count from Various Locations

• A query is made regarding the total number of logs originating from all countries except France, highlighting the need for comparative data analysis.

• The total number of logs from France is calculated as 2814, demonstrating how to filter and analyze data based on geographical parameters.

Specific Log Retrieval

• Inquiry into the number of logs associated with a specific IP reveals that there are only 14 events linked to it. This emphasizes targeted log retrieval techniques.

Visualization Techniques

• Introduction to visualization tools for better representation of log data. This includes creating visual dashboards for easier interpretation.

• Users can customize their views by adjusting display settings for source IP addresses within the visualization tool.

Dashboard Creation and Management

• Discussion on saving filters used in searches to create a dashboard named "VPN Logs," which allows users to manage their data more effectively.

• Upon saving the dashboard, users can view it directly, showcasing how saved filters translate into actionable insights.

Exporting Data

• Users have the option to export logs in PDF format for further analysis or reporting purposes. This feature enhances usability by allowing offline access to important data.

Editing and Formatting Options

• There are options available for editing titles and descriptions within the dashboard interface, providing flexibility in managing log information effectively.

Understanding the Process of Data Collection and Analysis

Overview of Data Handling

• The speaker introduces the concept of creating reports based on a dashboard, indicating that further discussions will occur in future sessions.

• The process begins with "Collecting" logs, which involves gathering data for analysis.

• After collection, the next step is "Normalize," where logs are organized and prepared for analysis.

• The speaker mentions building rules that help detect anomalies based on the analyzed data, emphasizing the importance of logical operators in queries.

Components of Splunk

• Discussion includes understanding components of Splunk and how they interact within the system to facilitate data analysis.