Using the DataCore Predefined Logstash Configuration File

Explore this Page

Overview

This section explains how to use the provided Logstash configuration file to forward SANsymphony syslog messages to third‑party platforms such as Elastic Search and Splunk. It covers placing and customizing the configuration file, configuring output destinations, opening the necessary ports, and running Logstash for proper log processing. It also provides guidance on verifying syslog forwarding and viewing logs in Elastic Search and Splunk for monitoring and analysis.

Using the DataCore Predefined Logstash Configuration File with Logstash

This section provides step‑by‑step instructions for placing, customizing, and running the DataCore‑provided Logstash configuration file. It walks you through updating inputs and outputs, opening required ports, and executing Logstash to process and forward SANsymphony syslog messages for indexing and visualization in Elastic Search or Splunk.

Follow these steps to place, customize, and run the configuration file for proper log forwarding and visualization.

  1. Copy or place the selected configuration file into the Logstash configuration directory, example:
    Copy
    Command
    E:\logstash-8.17.2\config\DataCore_RFC5424_verbose_doc_1.conf
  2. Confirm that the UDP port used in the configuration file (default is 514) is open and allowed through the firewall on the Logstash server. You can also specify new Port in RFC configuration file and in SANsymphony Syslog Settings based on your firewall connections.
  3. Open a command prompt on the Logstash server, navigate to the Logstash bin directory, for example:
    Copy
    Command
    cd E:\logstash-8.17.2\bin
  4. Customize the file as needed by updating input parameters and destination details for syslog message forwarding.
    1. In the output section of the configuration file, specify where to forward the parsed syslog messages. To forward logs to Elastic Search, provide the following:
      1. hosts: IP address and port of the Elastic Search server.
      2. index: Destination index name (example: syslog_5424).
      3. user and password: Valid credentials to authenticate with Elastic Search.

        After setting up Elastic Search output, make sure to create an index pattern in Elastic Search that matches the index name (example: syslog_5424) to visualize data.

    2. If needed, you can also forward the same syslog messages to Splunk by adding a udp output block. For this, specify:
      1. host: IP address of the Splunk server.
      2. port: UDP port configured in Splunk to receive syslog messages.
    3. stdout { codec => rubydebug }: Outputs messages to the console for verification.
    4. Example configuration snippet (RFC5424 format):
      Copy
      Script
      # output: where to send this data
      output {

        # Send to Elastic Stack (Elastic Search)
        elasticsearch {
            hosts => "ELASTIC SEARCH_IP"
            index => "syslog_5424"
            user => "elastic"
            manage_template => false
        }

        # Simultaneously forward to Splunk via UDP
        udp {
            host => "SPLUNK_IP"
            port => 5140
        }

        # Local output for debugging
        stdout { codec => rubydebug }
      }
      5. The Elastic Search output sends SANsymphony syslog messages to the Elastic Stack (Elastic Search and Kibana) for indexing and visualization. The optional udp block forwards the same messages to Splunk, where the host specifies the IP address of the Splunk server, and port defines the UDP port that receives syslog messages.
  5. Save the updated configuration file before running Logstash to ensure changes are applied.
  6. Make sure UDP port 514 is open on the Logstash server firewall to receive syslog messages from SANsymphony.

    By default, UDP port 514 is specified in both the DataCore-provided Logstash configuration file (RFC format) and the SANsymphony Syslog Settings. You can change this port as needed, for example to avoid conflicts with other services or to comply with your network policies, but make sure the same port is configured on both ends and is allowed through the firewall to ensure syslog message flow.

  7. Run Logstash with the configuration file by executing:
    Copy
    Command
    logstash.bat -f "E:\logstash-8.17.2\config\DataCore_RFC5424_verbose_doc_1.conf"
  8. Wait for Logstash to start and confirm it is running without errors by monitoring the console output. You should see Logstash startup logs indicating it’s listening on UDP port 514.
  9. Verify the messages are received and processed by checking your target platform (Elastic Search or Splunk), or by observing Logstash console output if using stdout for debugging.

Verifying Syslog Message Forwarding (Optional)

Follow the below steps to validate the syslog message forwarding:

  1. In DataCore SANsymphony, navigate to Settings > Syslog Settings and confirm the correct syslog server IP and port are configured (example: 10.131.0.58:514).
  2. Click Test Syslog and ensure the status message reads “Successfully contacted the endpoint from all servers.”
  3. Use the Add-DcsLogMessage PowerShell cmdlet to manually trigger a Syslog message and verify delivery to the configured endpoint.
  4. On the Logstash server, ensure the listener is running and check the terminal for:
    1. The test message appears in the console output.
    2. The message is parsed and displayed in JSON format.
This confirms end-to-end message flow from DataCore SANsymphony to Logstash.

Viewing Syslog Messages in Elastic Search and Splunk

After configuring SANsymphony to forward syslog messages, you can view and analyze these logs in third-party platforms such as Elastic Search and Splunk. These tools provide advanced filtering, visualization, and alerting capabilities to help monitor system activity, troubleshoot issues, and maintain operational visibility. The following sections describe how to access and view the forwarded logs in each platform.

Viewing Logs in Elastic Search

Once syslog messages are indexed in Elastic Search, they can be visualized and explored using Kibana.

Ensure an index pattern matching the destination index (example: syslog_5424) is created in Elastic Search. Without this index pattern, no data will be visible in the Discover tab or Dashboards.

  1. Open the Discover tab in Kibana.
  2. Select the index pattern (for example: syslog_5424*) to view the received messages.
  3. Use the time filter to set the desired date and time range.
  4. Log entries appear with fields such as:
    1. @timestamp: Time of the log.
    2. appname: Name of the application (example: DataCore SANsymphony).
    3. hostname: Source system name.
    4. severity: Log level (example: INFO, ERROR).
    5. syslog_msg: Message content.
    You can filter or search specific log entries using field values or keywords to narrow down results.

Viewing Logs in Splunk

When syslog messages from SANsymphony are forwarded to Splunk, they can be viewed using the Search & Reporting app in Splunk Web.

  1. Ensure Splunk is configured to listen on the correct UDP port (example: udp: 514).
  2. Navigate to Settings > Add Data in Splunk Web.
  3. During the data input setup:
    1. Under Input Settings, create or select the appropriate Index (for example, datacore, as used in the sample query in Step 4).
    2. locate the Source Type field, use the filter to search for and select syslog (described as: "Output produced by many syslog daemons, as described in RFC3164 by the IETF").
    3. Optionally, you may choose Operating System as the category.
  4. After configuring the data input, go to Search & Reporting and run the following query to view the logs.
    Copy
    Command
    source="udp:514" index="datacore" sourcetype="syslog"

You can further refine your search using keywords, field names, or create custom dashboards to visualize patterns or alerts.

Learn More