Release notes

7.17.5-27.4

Added a new REST API _siren/nodes/benchmark that can be used to evaluate different aspects of the Federate plugin.

Implemented Elasticsearch 7.17.5 compatibility.
Improved networking performance of the search-project task.
Updated default values for the following settings:
- The setting siren.io.pipeline.hash.partitions_per_node that sets the number of partitions per node used in the HASH_JOIN algorithm, now defaults to the number of CPUs - 1.
- The setting siren.io.pipeline.max_packet_size that sets the size of data packets sent between nodes of the cluster now defaults to 1MB.

Fixed the retrieval of dynamic and node-level settings siren.io.pipeline.* in Federate tasks.

Fixed caching issue of selected shards for an index by listening to index routing changes.
Fixed handling of OutOfMemoryError errors when serializing data packets.
Fixed a join issue with virtual indices and keyword join fields.

Upgraded to Elasticsearch 7.17.1
Various optimizations of the vectorized processing pipeline. Most noticeably:
- In case of local transfer requests, data packets serialization now directly uses arrowBuf.
- Improved vectorization for projection of single-valued columns.

Upgraded to Elasticsearch 7.17.0
Vectorization of distributed hash semi-join.
Vectorization of distributed inner join.
Added support to join on _id metadata field for semi-join queries.
- Limitation: projection of _id metadata field is not supported in the context of an inner join.
Added support for a placeholder to represent the Elasticsearch configuration directory in the JDBC properties of a datasource configuration.

Removed a deprecated property, siren.io.tuple.collector.packet_size, in favor of the siren.io.pipeline.max_packet_size setting.
Removed a deprecated property, siren.io.tuple.collector.hash.partitions_per_node, in favor of the siren.io.pipeline.hash.partitions_per_node setting.
Removed a deprecated property, siren.io.tuple.collector.number_of_nodes, in favor of the siren.io.pipeline.number_of_nodes setting.