HTTP event listener#

The HTTP event listener plugin allows streaming of query events, encoded in JSON format, to an external service for further processing, by POSTing them to a specified URI.

Rationale#

This event listener is a simple first step into better understanding the usage of a datalake using query events provided by Trino. These can provide CPU and memory usage metrics, what data is being accessed with resolution down to specific columns, and metadata about the query processing.

Running the capture system separate from Trino reduces the performance impact and avoids downtime for non-client-facing changes.

Requirements#

You need to perform the following steps:

Provide an HTTP/S service that accepts POST events with a JSON body.
Configure http-event-listener.connect-ingest-uri in the event listener properties file with the URI of the service.
Detail the events to send in the Configuration section.

Configuration#

To configure the HTTP event listener plugin, create an event listener properties file in etc named http-event-listener.properties with the following contents as an example:

event-listener.name=http
http-event-listener.log-created=true
http-event-listener.connect-ingest-uri=<your ingest URI>

And set add etc/http-event-listener.properties to event-listener.config-files in Config properties:

event-listener.config-files=etc/http-event-listener.properties,...

Configuration properties#

Property name	Description	Default
http-event-listener.log-created	Enable the plugin to log `QueryCreatedEvent` events	`false`
http-event-listener.log-completed	Enable the plugin to log `QueryCompletedEvent` events	`false`
http-event-listener.log-split	Enable the plugin to log `SplitCompletedEvent` events	`false`
http-event-listener.connect-ingest-uri	The URI that the plugin will POST events to	None. See the requirements section.
http-event-listener.connect-http-headers	List of custom HTTP headers to be sent along with the events. See Custom HTTP headers for more details	Empty
http-event-listener.connect-retry-count	The number of retries on server error. A server is considered to be in an error state when the response code is 500 or higher	`0`
http-event-listener.connect-retry-delay	Duration for which to delay between attempts to send a request	`1s`
http-event-listener.connect-backoff-base	The base used for exponential backoff when retrying on server error. The formula used to calculate the delay is `attemptDelay = retryDelay * backoffBase^{attemptCount}`. Attempt count starts from 0. Leave this empty or set to 1 to disable exponential backoff and keep constant delays	`2`
http-event-listener.connect-max-delay	The upper bound of a delay between 2 retries. This should be used with exponential backoff.	`1m`
http-event-listener.*	Pass configuration onto the HTTP client

Custom HTTP headers#

Providing custom HTTP headers is a useful mechanism for sending metadata along with event messages.

Providing headers follows the pattern of key:value pairs separated by commas:

http-event-listener.connect-http-headers="Header-Name-1:header value 1,Header-Value-2:header value 2,..."

If you need to use a comma(,) or colon(:) in a header name or value, escape it using a backslash (\).

Keep in mind that these are static, so they can not carry information taken from the event itself.