Auditing user sessions

If you want to perform internal audits, you can enable the session audit feature. This feature allows you to log the actions that are performed by the logged-in user.

Configuring session logs

To enable the session audit feature, set the enabled parameter to true in the following code in the investigate.yml file.

siren_audit:
  enabled: true                       # The value is false by default.
  temporary_logs_folder: './optimize' # The folder where the system can store the unsaved entries during shutdown or restart.
                                      # The default value is the './optimize' folder.
  logged_types:                       # The type of data that is logged.
    - ui                              # The default value is an array that contains all possible types: ui, data, savedObjects, acl, and count.
    - search
    - savedObject
    - count
    - dataExport
  logged_saved_object_types:          # By default, enable logging on all saved object types.
    'graph',
    'relational-graph'
    'visualization',
    'index-pattern',
    'ontology-model',
    'config',
    'dashboard',
    'dashboardgroup',
    'eid',
    'relation',
    'template',
    'datasource',
    'script',
    'search',
    'url',
    'doc',
    'fingerprint',
    'mixed-state-docs',
    'sirenapiscript',
    'sidebaroptions'
  logged_saved_object_body_types: []  # Types of saved objects to log the body, before and after operation. By default, the value is empty.
                                      # You can use any of the types that are listed under `logged_saved_object_types`.
                                      # Note: Be aware that the 'graph' type can contain sensitive information in the body.
                                      # The body of a graph can be very large, depending on the number of nodes it contains.
                                      # For example, 100 nodes is considered a small object, 5k nodes is a large object, and 20k nodes is a very large object.
  log_response_body: false            # If the value is set to 'true', the system logs the response body for Elasticsearch search requests and saved objects requests (excluding responses that contain the body of the object).
  log_header_names: []                # Array of header names to be logged in addition to all headers with "siren-" prefix which are always logged when audit is enabled
  log_remote_ip_from_header_name: 'my-header' # By default Investigate will check for x-forwarded-for header and extract the first ip address from it
                                              # Sometimes a non standard header is used to store the client ip address
                                              # In such case this property allows to specify such header name
  wait_for_outputs_initialization: false      # When set true Investigate will not start until all configured outputs are initialized
  # If you do not define the outputs, the system uses the main Elasticsearch cluster to store audit entries.
  # It is possible to configure more than one remote Elasticsearch output.
  outputs:
    - elasticsearch
        manage_template: true          # default true
        max_retries: 3                 # default 3
        queue_size: 1000               # default 1000
        flush_interval: 10000          # default 10000 ms
        flush_size: 50                 # default 50
        index_prefix: 'siren-audit-'   # default siren-audit-
        index_interval: daily|monthly  # default monthly
        number_of_shards: 1            # default 1
        number_of_replicas: 1          # default 1
        url: https://host:port
        username: username
        password: password
        ssl:
          verificationMode: none | certificate | full  # default full
          certificateAuthorities: [ "/path/to/your/CA.pem" ]
          certificate: /path/to/your/client.crt
          key: /path/to/your/client.key
          keyPassphrase: changeme

# When set, it is written to the instanceName field of audit entries
# Useful when multiple Investigate instances are configured to log audit entries to the same Elasticsearch cluster
investigate_core.instance_name: "my-instance-01"

Session log indices

After the feature is enabled, the system automatically creates an index that stores the audit entries. If you use the default value of elasticsearch as the output type, the system creates the index by using the output configuration credentials. If no output is defined, the system creates the index on the main Elasticsearch cluster by using the Siren Investigate System credentials.

Make sure that either the remote cluster user or the Siren Investigate Administrator user has the following security permissions:

  • Can store an index template.

  • Can create indices with a name starting with the configured prefix, for example, the default prefix is siren-audit-.

  • Can write to those indices.

Session log entries

The system can log the following types of session data:

  • ui: Actions that are performed by the user as they are interacting with the application. For example, a log entry is created when a user opens a dashboard.

  • savedObject: Access to system configuration objects. For example, a log entry is created when a user requests a configuration object for particular dashboard.

  • search: All data-related search queries. For example, a log entry is created when a visualization on a dashboard is fetching the data to render.

  • count: All queries that obtain the document count. For example, a log entry is created when a query is used to fetch the count of documents visible on a dashboard.

  • dataExport: A log entry is created each time a user attempts to export data as JSON or CSV.

If the logged_saved_object_body_types option contains a graph, it can log potentially sensitive information in the audit entry. This happens because saved objects of type graph can contain copies of fields from documents in the Elasticsearch indices.

When the log_response_body option is set to true, you must use a different instance of Siren Investigate to view the session audit logs - one that has auditing disabled. If the same instance is used to view the logs, the log entries will quickly grow in size, causing errors. This is due to recursion; a request logs a response, which when viewed again, again logs a response.

It is not currently possible to monitor the status of the audit index. In the event that the output remote cluster is down, the system can accumulate the log entries in a back-end, in-memory queue. If the queue reaches its maximum size, the system starts to lose the oldest audit entries. To avoid data loss, system administrators must actively monitor the health of remote audit clusters to keep them online.

For all 10.5.x versions of Investigate the Elasticsearch used as an output for session audit logs must be >= 6.8.0