You can explore your data interactively from the Discover page. You have access to every document in every index which matches the selected index pattern. You can submit search queries, filter the search results, and view document data. You can also see the number of documents that match the search query and get field value statistics. If a time field is configured for the selected index pattern, the distribution of documents over time is displayed in a histogram at the top of the page.
The time filter restricts the search results to a specific time period. You can set a time filter if your index contains time-based events and a time-field is configured for the selected index pattern.
Because of the variety of date formats in use, and the possibility that an index is fixed to use a specific format, Siren provides sample valid dates when editing filters.
By default, the time filter is set to the last 15 minutes. You can use the Time Picker to change the time filter or select a specific time interval or time range in the histogram at the top of the page.
To set a time filter with the Time Picker:
Click Time Picker () in the Siren Investigate toolbar.
To set a quick filter, click one of the shortcut links.
Click Select All in the Apply to Dashboards section to apply the time filter to all dashboards. Alternatively, you can select individual dashboards. The time filter is applied to the current dashboard by default.
To specify a time filter relative to the current time, click Relative and specify the start time as a number of seconds, minutes, hours, days, months or years. You can also specify the end time relative to the current time. Relative times can be in the past or the future.
To specify both the start and end times for the time filter, click Absolute and select a start and end date. You can adjust the time by editing the To and From fields.
Click the caret () in the bottom right corner to close the Time Picker.
To set a time filter from the histogram, do one of the following:
Click the bar that represents the time interval you want to zoom in on.
Click and drag to view a specific time span. You must start the selection with the cursor over the background of the chart. The cursor changes to a plus sign (+) when you move the mouse pointer over a valid start point.
To move forward or backward in time, click the arrows to the left or right of the Time Picker:
You can use your browser’s Back button to undo your changes.
The displayed time range and interval are shown on the histogram. By default, the interval is set automatically based on the time range. To use a different interval, click the link and select an interval.
You can search the indices that match the current index pattern by entering your search criteria in the Query bar. You can perform a simple text search, use the Lucene query syntax, or use the full JSON-based Elasticsearch Query DSL.
When you submit a search request, the histogram, Documents table, and Fields list are updated to reflect the search results. The total number of hits (matching documents) is shown in the toolbar. The Documents table shows the first five hundred hits. By default, the hits are listed in reverse chronological order, with the newest documents shown first. You can reverse the sort order by clicking the Time column header. You can also sort the table by the values in any indexed field. For more information, see Viewing document data.
To search your data, enter your search criteria in the Query bar and press Enter or click Search () to submit the request to Elasticsearch.
To perform a free text search, enter a text string. For example, if you are searching web server logs, you could enter
safarito search all fields for the term
To search for a value in a specific field, prefix the value with the name of the field. For example, you could enter
status:200to find all of the entries that contain the value
200in the status field.
To search for a range of values, you can use the bracketed range syntax,
[START_VALUE TO END_VALUE]. For example, to find entries that have
4xxstatus codes, you could enter
status:[400 TO 499].
To specify more complex search criteria, you can use the Boolean operators
NOT. For example, to find entries that have
4xxstatus codes and have an extension of
html, you could enter
status:[400 TO 499] AND (extension:php OR extension:html).
|These examples use the Lucene query syntax. You can also submit queries using the Elasticsearch Query DSL. For examples, see query string syntax in the Elasticsearch Reference.
Saving searches enables you to reload them into Discover and use them as the basis for visualizations. Saving a search saves both the search query string and the currently selected index pattern.
To save the current search:
Click Save in the Siren Investigate toolbar.
Enter a name for the search and click Save.
You can import, export and remove saved searches from.
To load a saved search into Discover:
Click Open in the Siren Investigate toolbar.
Select the search you want to open.
If the saved search is associated with a different index pattern than is currently selected, opening the saved search also changes the selected index pattern.
When you submit a search request, the indices that match the currently-selected index pattern are searched. The current index pattern is shown below the toolbar. To change which indices you are searching, click the index pattern and select a different index pattern.
For more information about index patterns, see Creating an Index Pattern Search.
As more documents are added to the indices you are searching, the search results shown in Discover and used to display visualizations get stale. You can configure a refresh interval to periodically resubmit your searches to retrieve the latest results.
To enable auto refresh:
Click Time Picker () in the Siren Investigate toolbar.
Click Auto refresh.
Choose a refresh interval from the list.
When auto refresh is enabled, the refresh interval is displayed next to the Time Picker, together with a Pause button. To temporarily switch off auto refresh, click Pause.
|If auto refresh is not enabled, you can manually refresh visualizations by clicking Refresh.
You can filter the search results to display only those documents that contain a particular value in a field. You can also create negative filters that exclude documents that contain the specified field value.
You can apply field filters in the following ways:
From the Selected fields list
From the Documents table
By manually adding a filter
The applied filters are shown below the Query bar. Negative filters are shown in red.
To add a filter from the Selected fields list:
Click the name of the field you want to filter on. This displays the top five values for that field.
Click one of the following filters:
The Filter for value () button includes only those documents that contain that value in the field.
The Filter out value () button excludes documents that contain that value in the field.
To add a filter from the Documents table:
Click the Expand icon to the left of the document’s table entry.
In the expanded view, click one of the following filters:
Filter for value includes only those documents that contain that value in the field.
Filter out value excludes documents that contain that value in the field.
Toggle column in table allows you to hide or show a field as a column in the table.
Filter for field present includes only those documents that contain the field.
Filter for more like this includes documents that contain similar text to the text in that field.
To manually add a filter:
Click Add a filter. A popup will be displayed for you to create the filter.
Choose a field to filter by. This list of fields will include fields from the index pattern you are currently querying against.
Choose an operation for your filter.
The following operators can be selected:
Filter where the value for the field matches the given value.
Filter where the value for the field does not match the given value.
is one of
Filter where the value for the field matches one of the specified values.
is not one of
Filter where the value for the field does not match any of the specified values.
Filter where the value for the field is in the given range.
is not between
Filter where the value for the field is not in the given range.
Filter where any value is present for the field.
does not exist
Filter where no value is present for the field.
Choose the value(s) for your filter.
(Optional) Specify a label for the filter. If you specify a label, it will be displayed below the Query bar instead of the filter definition.
Click Save. The filter will be applied to your search and be displayed below the Query bar.
To make the filter editor more user-friendly, you can enable the
filterEditor:suggestValues advanced setting. Enabling this will
cause the editor to suggest values from your indices if you are
filtering against an aggregatable field. However, this is not
recommended for extremely large data sets, as it can result in long
To modify a filter, move the moue pointer over it and click one of the action buttons.
- Enable Filter
Switch off the filter without removing it. Click again to switch the filter on again. Diagonal stripes indicate that a filter is switched off.
- Pin Filter
Pin the filter. Pinned filters persist when you switch contexts in Siren Investigate. For example, you can pin a filter in Discover and it remains in place when you switch to Visualize. Note that a filter is based on a particular index field—if the indices being searched do not contain the field in a pinned filter, it has no effect.
- Invert Filter
Switch from a positive filter to a negative filter and vice-versa.
- Remove Filter
Remove the filter.
- Edit Filter
Edit the filter definition. Enables you to manually update the filter and specify a label for the filter.
To apply a filter action to all the applied filters, click Actions and select the action.
You can edit a filter by changing the field, operator, or value associated with the filter (see the Add Filter section), or by directly modifying the filter query that is performed to filter your search results. This enables you to create more complex filters that are based on multiple fields.
To edit the filter query, first click Edit for the filter, then click Edit Query DSL.
You can then edit the query for the filter.
For example, you could use a bool query to create a filter for the sample log data that displays the hits that originated from Canada or China that resulted in a 404 error:
When you submit a search query, the 500 most recent documents that match
the query are listed in the Documents table. You can configure the
number of documents shown in the table by setting the
discover:sampleSize property in
By default, the table shows the localized version of the time field
configured for the selected index pattern and the document
fields to the Documents table from the Fields list. You can
sort the listed
documents by any indexed field that’s included in the table.
To view a document’s field data, click the Expand icon to the left of the document’s table entry.
To view the original JSON document (pretty-printed), click the JSON tab.
To display or hide a field’s column in the Documents table, click Toggle column in table ().
To collapse the document details, click Collapse ().
You can sort the documents in the Documents table by the values in any indexed field. If a time field is configured for the current index pattern, the documents are sorted in reverse chronological order by default.
To change the sort order, move the mouse pointer over the name of the field you want to sort by and click the sort button. Click again to reverse the sort order.
By default, the Documents table shows the localized version of the time
field that’s configured for the selected index pattern and the document
_source. You can add fields to the table from the Fields list or
from a document’s field data.
To add a field column from the Available fields list, move the mouse pointer over the field and click Add.
To add a field column from a document’s field data, expand the document and click the field’s Toggle column in table () button.
Added field columns replace the
_source column in the Documents
table. The added fields are also added to the
Selected fields list.
To rearrange the field columns, move the mouse pointer over the header of the column you want to move and click the Move left or Move right button.
For certain applications it can be useful to inspect a window of documents surrounding a specific event. The context view enables you to do just that for index patterns that are configured to contain time-based events.
To show the context surrounding an anchor document, click the Expand icon to the left of the document’s table entry and then click the View surrounding documents link.
The context view displays a number of documents before and after the anchor document. The anchor document itself is highlighted in blue. The view is sorted by the time field specified in the index pattern configuration and uses the same set of columns as the Discover view the context was opened from. If there are multiple documents with the same time field value, the internal document order is used as a secondary sorting criterion by default.
The field used for tie breaking in case of equal time field values can
be configured using the advanced setting
in , which defaults to the
_doc field. The value of this setting can be a comma-separated list
of field names, which will be checked in sequence for suitability when a
context is about to be displayed. The first suitable field is then used
as the tie breaking field. A field is suitable if the field exists and
is sortable in the index pattern the context is based on.
While not required, you should only use fields which have doc values enabled to achieve good performance and avoid unnecessary field data usage. Common examples for suitable fields include log line numbers, monotonically increasing counters and high-precision timestamps.
The number of documents displayed by default can be configured using the
context:defaultSize setting in .
You can change the number documents displayed before and after the anchor document independently.
To increase the number of displayed documents that are newer than the anchor document, click Load 5 more above the document list or enter the desired number into the input box.
To increase the number of displayed documents that are older than the anchor document, click Load 5 more below the document list or enter the desired number into the input box
The default number of documents loaded with each click can be configured
context:step setting in ManagementAdvanced Settings.
Depending on how the documents are partitioned into index patterns, the context view may contain many documents not related to the event under investigation. To adapt the focus of the context view to the task at hand, you can use filters to restrict the documents considered by Siren Investigate for display in the context view.
When switching from the Discover view to the Context view, the previously applied filters are carried over. Pinned filters remain active while normal filters are copied in a switched off state. You can selectively re-enabled them to refine your context view.
New filters can be added using the Add a filter link in the filter bar, by clicking the filter icons appearing when moving the mouse pointer over a field, or by expanding documents and clicking the filter icons in the table.
From the Fields list, you can see how many of the documents in the Documents table contain a particular field, what the top five values are, and what percentage of documents contain each value.
To view field data statistics, click the name of a field in the Fields list.
Selected Fields are displayed on the Selected Field list at the top left of the Discover page.
Click a field to see the field’s Viewing field data statistics.
To remove a field, click the field and click Remove.
After you have selected at least one field, you can then click Generate Dashboard to begin auto generating a new Dashboard.
If you have not selected any fields, you can allow Siren Investigate to select the fields it believes are the most relevant by clicking Autoselect Most Relevant.
|This button is only visible if there are no fields selected. If you have selected fields, the Generate Dashboard button will be visible in its place.
Siren Investigate begins testing the fields with each field being analyzed for relevance against a number of heuristics, for example, whether all values are unique - which indicates a potential ID field and unlikely to be relevant for visualizations.
After all the fields have been analyzed, a test report is displayed.
This report shows all the fields in the Discover data, which fields are selected as most relevant, the field type, the visualization selected for that type, the relevancy score and any notes on why the field was or was not selected as a relevant field.
You can add and remove fields you would like selected using the check boxes on the left.
When you are ready, click Ok to select the fields.
After the fields have populated the Selected Fields list, you are ready to generate a dashboard.