Set Up Security

The Siren Federate plugin is compatible with Search Guard and Elastic Stack Security. You will find below instructions on how to configure both solutions for Federate.

Search Guard

We assume in this section that you are familiar with Search Guard, that Search Guard is installed in your cluster, and that you know how to configure users, roles and permissions. If not, please refer to the Search Guard documentation first.

Configuring Action Groups

Here is a list of action groups that are suitable for Federate.

sg_action_groups.yml
###### INDEX LEVEL ######

INDICES_ALL:
  allowed_actions:
  - "indices:*"

MANAGE:
  allowed_actions:
  - "indices:monitor/*"
  - "indices:admin/*"                                           (1)

WRITE:
  allowed_actions:
  - "indices:data/write*"
  - "indices:admin/mapping/put"

READ:                                                           (2)
  allowed_actions:
  - "indices:data/read*"

VIEW_INDEX_METADATA:                                            (3)
  allowed_actions:
  - "indices:admin/aliases/get"
  - "indices:admin/aliases/exists"
  - "indices:admin/get"
  - "indices:admin/exists"
  - "indices:admin/mappings/fields/get*"
  - "indices:admin/mappings/get*"
  - "indices:admin/mappings/federate/connector/get*"
  - "indices:admin/mappings/federate/connector/fields/get*"
  - "indices:admin/types/exists"
  - "indices:admin/validate/query"
  - "indices:monitor/settings/get"

###### CLUSTER LEVEL ######

CLUSTER_ALL:
  allowed_actions:
  - "cluster:*"

CLUSTER_MONITOR:
  allowed_actions:
  - "cluster:monitor/*"                                         (4)

CLUSTER_COMPOSITE_OPS:
  allowed_actions:
  -  CLUSTER_COMPOSITE_OPS_RO
  - "indices:data/write/bulk"

CLUSTER_COMPOSITE_OPS_RO:
  allowed_actions:
  - "indices:data/read/mget"
  - "indices:data/read/msearch"
  - "indices:data/read/mtv"
  - "indices:data/read/scroll*"

CLUSTER_MANAGE:                                                 (5)
  allowed_actions:
  - CLUSTER_INTERNAL_FEDERATE
  - "cluster:admin/federate/*"
  - "indices:admin/aliases*"

CLUSTER_INTERNAL_FEDERATE:                                      (6)
  allowed_actions:
  - "cluster:internal/federate/*"

  1. Federate’s actions related to index administration are prefixed with indices:admin/federate

  2. Federate’s actions related to index read are prefixed with indices:data/read/federate

  3. Grants permission to read index metadata, like getting field mapping

  4. Federate’s actions related to cluster monitoring are prefixed with cluster:monitor/federate

  5. Federate’s actions related to Federate administration are prefixed with cluster:admin/federate

  6. All internal Federate’s actions are prefixed with cluster:internal/federate

Configuring Role-Based Access Control

Given the action groups defined above, we can define two types of roles:

  • the federate_admin role which can administrate Federate. For example, this role can manage license, virtual indices, ingestion jobs, etc.

  • the federate_user role with read-only permissions which can execute Federate’s search requests against one or more indices (virtual or not).

sg_roles.yml
federate_admin:
  cluster_permissions:
    - CLUSTER_MANAGE                    (1)
    - CLUSTER_MONITOR                   (2)
  index_permissions:
    - index_patterns:
      - 'logstash-*'
      allowed_actions:
        - MANAGE
        - READ
        - VIEW_INDEX_METADATA           (3)

federate_user:
  cluster_permissions:
    - CLUSTER_INTERNAL_FEDERATE         (4)
  index_permissions:
    - index_patterns:
      - 'companies'
      allowed_actions:
        - READ
        - VIEW_INDEX_METADATA           (3)

  1. Grants Federate cluster administration permissions.

  2. Grants Federate cluster monitoring permissions.

  3. Grants permissions to read index metadata. This is required given that the Federate’s query engine will access index schema metadata using indices:admin/mappings/fields/get during the query evaluation.

  4. Grants cluster-level permission for Federate’s internal actions. This is required for every Federate users.

Securing Connector

When using Search Guard, Federate will need to authenticate as a user with all the permissions on the indices storing datasources and virtual indices configuration. The credentials of this user can be specified through the following node configuration settings:

  • siren.connector.username: the username of the Federate system user.

  • siren.connector.password: the password of the Federate system user.

Federate system role

If your cluster is protected by Search Guard, it is required to define a role with access to the Federate indices and internal operations and to create a Federate system user with this role.

Whenever a virtual index is created the Federate plugin creates a concrete Elasticsearch index with the same name as the virtual index: when starting up, the Federate plugin will check for missing concrete indices and will attempt to create them automatically.

sg_roles.yml
federate_system:
  index_permissions:
    - index_patterns:
      - '?siren-federate-*'
      allowed_actions:
        - INDICES_ALL

Then create a user with that role e.g., a user called federate_system_user.

Master node in a cluster with authentication and federate_system_user user:

=== .elasticsearch.yml

siren.connector.username: federate_system_user
siren.connector.password: password
siren.connector.encryption.secret_key: "1zxtIE6/EkAKap+5OsPWRw=="

===

Example 1. JDBC node in a cluster with authentication and federate_system_user user:
elasticsearch.yml
siren.connector.username: federate_system_user
siren.connector.password: password
siren.connector.encryption.secret_key: "1zxtIE6/EkAKap+5OsPWRw=="
node.attr.connector.jdbc: true

Restart the nodes after setting the appropriate configuration parameters.

Administrative role

In order to manage, search, read datasources and virtual indices, it is required to grant the following cluster and indices-level permissions:

  • cluster:admin/federate/connector/* which are given by the CLUSTER_MANAGE group;

  • indices:admin/federate/connector/* which are included in the MANAGE group;

  • indices:admin/mappings/federate/connector/* which are part of the VIEW_INDEX_METADATA group; and

  • indices:data/read/federate/connector/* which are part of the READ group.

When a virtual index is defined, index-level write permissions are required because Federate creates a concrete index with the same name for interoperability with authentication plugins, unless such an index already exists.

For instance, if a MySQL datasource is defined and is named db_mysql, an index named db_mysql will be created. Then, the following connector_admin role can be created in order to manage/read/search it.

sg_roles.yml
connector_admin:
  cluster_permissions:
    - CLUSTER_MANAGE
    - CLUSTER_MONITOR
  index_permissions:
    - index_patterns:
      - 'db_mysql'
      allowed_actions:
        - READ
        - VIEW_INDEX_METADATA
        - MANAGE

Write operations are made on the virtual index, not against the actual datasource per se.

Search role

In order to search virtual indices, a user needs indices:data/read/federate/connector/* permissions which are part of the READ group.

Keeping with the db_mysql virtual index example, a connector_user needs the following permissions granted:

sg_roles.yml
connector_user:
  cluster:
    - CLUSTER_INTERNAL_FEDERATE
  index_permissions:
    - index_patterns:
      - 'db_mysql'
      allowed_actions:
        - READ

Elastic X-Pack Security

https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-security.html

federate system
{
  "federate_system": {
    "cluster": [
      "cluster:internal/federate/*",
      "cluster:admin/federate/*",
      "cluster:monitor/*"
    ],
    "indices": [
      {
        "names": [
          "/\\.siren.*/"
        ],
        "privileges": [
          "all"
        ]
      },
      {
        "names": [
          "*"
        ],
        "privileges": [
          "indices:monitor/*",
          "indices:admin/*",
          "indices:data/read*",
          "indices:data/write*"
        ]
      }
    ]
  }
}
Admin role
{
  "federate_admin": {
    "cluster": [
      "cluster:internal/federate/*",
      "cluster:admin/federate/*",
      "cluster:monitor/*",
      "cluster:admin/xpack/security/*"
    ],
    "indices": [
      {
        "names": [
          "*"
        ],
        "privileges": [
          "indices:monitor/*",
          "indices:admin/*",
          "indices:data/read*"
        ]
      }
    ]
  }
}
Search role
{
  "federate_user": {
    "cluster": [
      "cluster:internal/federate/*"
    ],
    "indices": [
      {
        "names": [
          "logstash-*"
        ],
        "privileges": [
          "indices:data/read*",
          "indices:admin/aliases/get",
          "indices:admin/aliases/exists",
          "indices:admin/get",
          "indices:admin/exists",
          "indices:admin/mappings/fields/get*",
          "indices:admin/mappings/get*",
          "indices:admin/mappings/federate/connector/get*",
          "indices:admin/mappings/federate/connector/fields/get*",
          "indices:admin/types/exists",
          "indices:admin/validate/query",
          "indices:monitor/settings/get",
          "indices:admin/template/get"
        ]
      }
    ]
  }
}