/
Logs

Logs

Logs allow you to see detailed information about how a stream execution went, search for errors, and form an understanding of the timeline of events. The Logs page provides you with the details of the events generated every time you run a stream.

Note!

There can be some cases where no logs will be displayed, for example, HTTP Functions where only errors will be shown.  

Opening the Logs page

You can open the Logs page by clicking View logs in the Stream editor.

View logs toolbar.png

Alternatively, when in the Streams tab, under the Actions column of a stream, click the More drop-down, then click Log.

The Logs page

You see the following information in the Logs page:

Column

Description

Column

Description

Timestamp

The date and time the event was generated. The timestamp is displayed in the YYYY-MM-DD HH:mm:ss:fff format.

Note!

The timestamp is displayed according to the browser’s local timezone.

Severity

Indicates the event type:

  • Information

  • Warning

  • Error

Origin

The origin of the event, namely which function generated the event. This column displays ‘Engine’, ‘Data correction’, ‘Stream audit’, and ‘Tx manager’ for logs that do not come from a particular function.

Execution ID

The Execution ID of the event.

Message

The message generated for the event.

Note!

If the message is larger than 4000 characters, it is truncated. 

Click on a log entry to expand it and view more information about the event, such as the Execution ID and the complete message of the event. Click the clipboard icon to copy details such as the Execution ID.

You can select how many rows you can view on a page. The default setting is 100 rows per page. The current logs display limit is 10,000 log entries.

Logs page toolbar

You can control the Logs page with the options in the Logs page toolbar:

  • Filter

  • Display options

  • Refresh data

Refresh data

Click Refresh data to refresh the Logs page with updated logs.

Display options

Click Display options to open the Display options view and select which columns to display in the Logs page. Check the checkbox(es) of the column(s) to display, and click Apply.

Filter

Click the Filter option to open the Filter panel to display events based on certain criteria in the Logs page.

In the Filter page, you can filter the logs using all, or a combination, of the following criteria:

Criteria

Description

Criteria

Description

Search

Filters the logs by the message text.

The search bar uses the Elasticsearch regular expression and operators to match with results in the logs. See this Elasticsearch guide for more information on regular expression and syntax.

Some examples of how the search bar matches results:

  • Searching with the single-word expression fish filters the log to display any results that have the word “fish”, including “Fish & Chips” and “I love fish chips.”

  • Searching with the boolean OR expression fish chips filters the log to display any results that have the word "fish" or "chips" in any order, including "fish", "Chips", "I love fish chips", "Fish & Chips", and "I love bamboo & Fish." Note that the expression is not case-sensitive.

  • Searching with the boolean AND expression fish AND chipsfilters the log to display any results that have the word "fish" and "chips" in any order, including "I love fish chips", "Fish & Chips", and "I love chips and Fish I love almost as much." Note that the expression is not case-sensitive.

  • Searching with the quoted string expression "collectors have finished" filters the log to display results that only match the entire quoted phrase such as "All collectors have finished.”

  • Searching with the regular expression .*[fd]ish.*[0-9]+filters the log to display any results that only have the words "fish" or "dish" in lower case, in that order, and ends with a numeric value, such as "I love fish dish number 4.” Note that when using regular expressions, filtering is case sensitive.

When Advanced is toggled on, you may also filter by:

  • Execution ID

  • Replica ID

Date

Filters the logs from a drop-down of pre-defined time and day.

When Custom is selected, you may filter the logs specifying the From and To date and time in the YYYY-MM-DD HH:mm:ss format.

Severity

Filters the logs by the Severity type.

  • Information

  • Warning

  • Error

Check the check box of the Severity type to display in the Logs view.

Origin

The origin of the event, namely which function generated the event. Check the check box of the function type to display in the Logs view.

  • Collectors

  • Processors

  • Forwarders

When Advanced is toggled on, you may check the check box of the specific function in the stream to display in the Logs view.

Once you have set the combination of filter criteria, click Filter. The Logs view then filters the logs by the filter criteria.

You may adjust the filter criteria accordingly and click Filter again to re-filter to the new set of criteria in the Logs view.

When a filter is applied to the Logs view, it is indicated with ‘Filter applied’ on the Filter option.

If you want to reset the Filter view, click Clear filter.