Observability

Analytical events

Notifications

Dashboard	Settings Notifications
ThreatX API	`api.threatx.com/tx_api/v2/subscriptions`

You can configure users to receive notifications on various events relating to threats, rule matches, changes to the IP allow, deny, and block lists. Notifications are typically sent by email, but you can configure a webhook notification to another app, such as Slack.

You create notifications in the ThreatX user interface by navigating to Settings Notifications. You can add a notification or edit an existing notification.

Notification Settings

Name

A unique name to identify this notification. It may only contain the following characters:

Lowercase letters
Hyphens
Numbers

Enabled

Notifications are sent to the specified target as configured.

Event Subject Area

You can send all events or limit the notification to specific types of events:

All Event Subjects
Entity-related Events
Rule-related Events
WAF List (Blacklist, Blocklist, Whitelist)
Entry-related Events

Event Incident

If you specify a type of event in the Event Subject Area, you can further limit the type of event. However, no other selection is available if you choose Event Subject Area All Event Subjects .

Event Source

You can limit notifications to alerts from a specific ThreatX system component.

All Event Sources
Automatic, risk engine-initiated Events
Manual, API-initiated Events

Limit By Site

You can limit notifications to alerts for events that affect one of a list of sites. This is only available when you select one of the following two options:

Event Subject Area All Event Subjects
Event Subject Area Entity-related Events.

Limit By WAF List Type

|You can limit notifications to events for one or more IP lists. Only available when you select Event Subject Area WAF List Blacklist, Blocklist, Whitelist

Notification Targets: Method

Email
Webhooks

You will need to configure the app to receive the notifications.For example, you can send notifications to Slack as described in their Incoming Webhooks article.

ThreatX maintenance and system status

You can view and subscribe to notifications for scheduled maintenance windows and any issues that might impact your ThreatX services at ThreatX Status.

Receiving event logs

The ThreatX Log Emitter exports event logs from the ThreatX platform to your log receiver and SIEM.You can use the logs in your investigations and to trigger events in your chosen log management solution.

Features

The Log Emitter forwards full details for all block, match, and audit events.

The logs are pushed are in JSON lines format over a TCP connection that is encrypted, and optionally authenticated, over TLS.If the Log Emitter subscription becomes suspended, the Log Emitter service queues your logs for delivery upon successful re-connection, and periodically attempts to re-establish a connection.

In case a Log Emitter subscription becomes suspended, the Log Emitter service queues your logs for delivery upon successful re-connection, and periodically (every half hour) attempts to re-establish a connection.

Once the Log Emitter re-establishes a connection for a previously suspended subscription, all queued log events are sent to the configured receiver. If the Log Emitter subscription cannot be resumed after several retries, this might indicate a configuration error or log receiver error.

Configuring a Log Emitter

To receive logs, you create an instance of the Log Emitter and then encrypt the connection between the Log Emitter and your receiver.

Prerequisites for Log Emitter

You must whitelist the following IP address ranges:
- 169.44.76.160/28
- 169.61.156.0/28
- 158.85.41.64/27
Verify that your log receiver or SIEM can parse JSON lines
Allow inbound connections your log receiver’s listening port if it is behind a firewall
Validate that your log receiver’s hostname resolves to a public IP address.

A certificate is needed to encrypt the connection between the ThreatX Log Emitter service and your log receiver. You must generate a self-signed or CA signed certificate for the hostname where the log receiver receives logs in one of two ways:

A Simple configuration that sets up an encrypted TCP connection without authentication.
An Advanced configuration adds mutual server and client certificate authentication to your configuration

Simple configuration

Generate a new certificate

$ openssl req \
  -x509 \
  -days 365 \
  -nodes \
  -newkey rsa:4096 \
  -keyout logreceiver.key \
  -out logreceiver.crt

Use the logreceiver.key and logreceiver.crt files on your log receiver.

Be sure to configure your on-premises log receiver to accept TCP connections with TLS encryption.

Advanced configuration

You can further secure the TCP encrypted connection between the ThreatX Log Emitter and your log receiver by adding mutual server and client certificate authentication:

Use the Certificate Authority of your choice to create an SSL/TLS certificate and private key for the ThreatX Log Emitter.
Create a valid server certificate and key in PEM format to install on the on-premises log receiver. The the CN of the server certificate must match the public domain name of the log receiver.
Configure your on-premises log receiver to accept TCP connections with TLS encryption and client certificate authentication.

Adding a Log Emitter

Dasboard	Settings Log Emitter then click Add Log Emitter
ThreatX API	`api.threatx.com/tx_api/v2/subscriptions`

Configure the settings as described in the following table. Click Save when done.

Table 1. Log Emitter Settings
Setting	Value	Description
Name	Text input	Unique name to identify the Log Emitter.
Hostname	Text input	Host name of your log receiver.
Port	Text input	Port number that your log receiver listens on
Send Client SSL/TLS Credentials to Log Receiver	Check box	Check to upload your SSL/TLS certificate and key
Verify Log Receiver SSL/TLS Certificate	Check box	When checked, the Log Emitter verifies the SSL/TLS certificate provided by the log receiver before sending log data.
Enabled	Check box	Start sending logs to your log receiver.

Description of logs

BlockEvent Log Type

The BlockEvent log type provides full details on requests that were blocked by the ThreatX sensor.

BlockEvent Message Example

{
    "message": "www.examplesite.net/example_uri",
    "msg_type": "BlockEvent",
    "timestamp": "2020-12-18T14:05:52Z",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64;rv: 82.0)",
    "dst_host": "www.examplesite.net",
    "uri": "/example_uri",
    "args": "oneequals1--",
    "request_id": "d3f02fff5db4824d83d145fad1258959",
    "random_id": null,
    "tls_fingerprint": null,
    "cookie": null,
    "js_fingerprint": null
}

Table 2. BlockEvent Type Properties
Name	Details
`message`	Complete target path of the request, including hostname and URI.
`msg_type`	Request was blocked at the individual request level or due to the entity being blocked at the Risk level.
`timestamp`	UTC timestamp of the request
`user_agent`	UserAgent presented by the entity making the request
`ip`	IP address presented by the entity making the request
`dst_host`	Target hostname of the request
`uri`	Target path of the request
`args`	Arguments (if any) provided in the request in `www-url-encoded` form
`request_id`	Unique identifier assigned to each request by the ThreatX platform
`random_id`	Additional unique identifier assigned to an entity by the ThreatX platform. See the note.
`tls_fingerprint`	TLS fingerprint (if any) associated with the entity making the request
`js_fingerprint`	Additional Unique identifier assigned to an entity by the ThreatX platform. See the note.

MatchEvent Log Type

The MatchEvent log type provides full details on requests that matched custom or common rule definitions when examined by the sensor.

MatchEvent Message Example

{
    "message": "www.examplesite.net/example_uri",
    "msg_type": "MatchEvent",
    "timestamp": "2020-12-18T14: 05: 52Z",
    "request_id": "d3f02fff5db4824d83d145fad1258959",
    "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:82.0)",
    "matches": [
        {
            "id": 202202,
            "description": "SqlAnalyzer: SQLi detected in form/args, sql_ids: 1",
            "classification": "SqlInjection",
            "state": "Exploitation",
            "contrib_score": 100,
            "risk": 70,
            "blocking": true,
            "beta": false
        }
    ],
    "ip": "222.222.222.222",
    "dst_host": "www.examplesite.net",
    "uri": "/",
    "args": "oneequals1--",
    "status_code": 0,
    "ssl": false,
    "risk": 70,
    "request_method": "GET",
    "content_type": null,
    "content_length": 0,
    "response_length": null,
    "upstream_response_time": null,
    "postblock_event": false,
    "random_id": 0,
    "tls_fingerprint": null,
    "cookie": null,
    "js_fingerprint": 0,
    "created": "2020-12-18T14: 05: 52.174+00: 00"
}

Table 3. MatchEvent Type Properties
Field	Details
`message`	Complete target path of the request, including hostname and URI.
`msg_type`	`[MatchEvent]` Request matched a custom or common rule.
`timestamp`	UTC timestamp of the request
`request_id`	Unique identifier assigned to each request by the ThreatX platform.
`user_agent`	UserAgent header value presented by the Entity making the request.
`list<matches>`	Values following this field provide specific information about why the rule that the request’s behavior matched: `description` - Description or name of the matched rule. `classification` - Industry-defined classification of the attack described in the rule. `state` - Industry-defined goal of the attack described in the rule. `contrib_score` - Reserved for ThreatX internal use. `risk` - Amount of risk that the matched rule contributes to the requesting entity’s risk score. `blocking` - Rule blocked a request `[True]` or allowed the request `[False]`. `beta` - Reserved for ThreatX internal use.
`ip`	IP address presented by the entity making the request.
`dst_host`	Target hostname of the request.
`uri`	Target path of the request.
`args`	Arguments (if any) provided in the request in `www-url-encoded` form.
`status_code`	Status code that the request received from the upstream server.
`ssl`	Request was transmitted over an HTTPS connection `[True]` or an HTTP connection `[False]`.
`request_method`	Request method type `[GET]` or `[POST]`
`content_type`	MIME content type/subtype (if any) presented in the request.
`response_length`	Length in bytes (if any) that the request received from the upstream server.
`upstream_response_time`	Length of time in milliseconds (if any) that it took the upstream server to respond to the request.
`postblock_event`	Request was submitted after a risk-based block was applied to the entity `[True]`.
`random_id`	Additional Unique identifier assigned to an entity by the ThreatX platform. See the note.
`tls_fingerprint`	TLS fingerprint (if any) associated with the entity making the request.
`cookie`	Additional Unique identifier assigned to an entity by the ThreatX platform. See the note.
`js_fingerprint`	Additional Unique identifier assigned to an entity by the ThreatX platform.
`created`	Timestamp of the request

Returns a null value except when logging an interrogation event. For information on interrogation, contact the ThreatX SOC.

Troubleshooting the Log Emitter

The following procedures describe basic checks that you can perform while troubleshooting your Log Emitter configuration.

Send test logs

Send a test log to verify that the server and client certificates are correctly generated and installed by running the following openssl command.

If you do not see DONE at the end, there is an issue with network connectivity or with the server or client certificates.

Self-signed certificates

$ echo '{"message":"test1"}' | openssl s_client \
    -servername <logreceiver.yourdomain.com>    \
    -connect <logreceiver.yourdomain.com:12345>

Mutual certificate authentication

$ echo '{"message":"test1"}' | openssl s_client \
    -servername <logreceiver.yourdomain.com>    \
    -connect <logreceiver.yourdomain.com:12345> \
    -cert logemitterclient.crt                  \
    -key logemitterclient.key

Your log receiver must be able to receive TCP data at the hostname provided to the Log Emitter. This requires a publicly resolvable hostname.

Verify incoming TCP data

Use tcpdump to verify that the log receiver is receiving TCP traffic on the correct port.

$ `tcpdump` port 12345

Even if you see traffic via tcpdump, you still need to ensure that any host-based firewall, such as iptables, is configured to allow the incoming traffic.

Dump incoming logs to a file (Logstash)

In Logstash, you can create a file output so you can quickly see if it is receiving the logs from the Log Emitter. Add the following fragment to your Logstash configuration file and restart Logstash.

logstash-fragment.json

output {
    file {
        path => "/tmp/threatx-raw.log"
    }
}

Handshake failed error

If you use a self-signed certificate and you receive this error message:

the handshake failed: error 1416F086: SSL/TLS Routines:
tls_process_server_certificate:certificate verify
failed:../ssl/statem/statem...cint.c:1915:: self signed certificate

The Log Emitter Send Client SSL/TLS Credentials to Log Receiver option might be enabled. When this option is enabled, the Log Emitter uses the provided SSL/TLS credentials to authenticate itself to the log receiver, which will of course fail when using a self-signed certificate.

Resolve the issue by deselecting the Send Client SSL/TLS Credentials to Log Receiver. Then, click Restart Log Emitter.