Logstack List Filebeats: Understanding the Configuration Options

When it comes to configuring Cisco Umbrella's log collection, one of the most important aspects is understanding the fileset structure and its various components. In this article, we will delve into the details of the Logstack list filebeats, exploring the different options and settings that are available.

Understanding the Fileset Structure

The Cisco Umbrella fileset relies on a specific file path structure being followed. This ensures that logs can be correctly parsed and processed by Filebeat. It's essential to note that retrieving logs from a Cisco-managed S3 bucket is not currently supported. However, configuring Cisco Umbrella to log to a self-managed S3 bucket is possible by following the Cisco Umbrella User Guide and AWS S3 input documentation.

Filebeat S3 Input

The Cisco Umbrella fileset primarily focuses on reading CSV files from an S3 bucket using the Filebeat S3 input. This process causes both ECS (Elastic Common Schema) and custom fields under rsa to be added to the event. To control the addition of raw parser fields to the event, you can use the `add_fields` processor.

Time Zone Configuration

When processing logs from a host on a different time zone, it's essential to set the timezone using the `event.timezone` field. This field can also be overwritten with the original time zone information if required. To disable this conversion, you can remove the `event.timezone` field or specify a custom time zone.

UDP Port Configuration

The UDP port to listen for syslog traffic can be configured using the `syslog.port` setting. If left empty, Filebeat will choose an available UDP port automatically. Additionally, the `bind_host` setting allows you to specify the interface to listen to UDP-based syslog traffic.

Log Path Configuration

Filebeat uses a specific log path structure to determine where to fetch log files from. This can be configured using wildcards or a predefined level of subdirectories. For example, you can use `/path/to/log/*/*.log` to fetch all log files from the specified directory and its subfolders.

FTD Fileset

The Cisco FTD fileset maps Security Event Syslog Messages to the Elastic Common Schema (ECS). This allows for easy integration with other security tools and provides a standardized way of processing log data. The FTD fileset primarily supports parsing IPv4 and IPv6 access list logs, including forwarded events.

In conclusion, configuring Cisco Umbrella's Logstack list filebeats requires a thorough understanding of the various options and settings available. By following this article, you should now have a better grasp of how to configure your Filebeat setup for optimal log collection and processing. Remember to carefully consider each setting to ensure that your logs are correctly parsed and processed for analysis and reporting purposes.