How do I know if my integration is functioning?

It is important to understand that the integrations you have configured are working correctly and sending telemetry to the SamurAI platform.

Integration Health

You can easily get an overview of which of your integrations are not healthy (e.g the integration status is not reported as OK) by viewing the Integrations Table or Telemetry Monitoring. Telemetry Monitoring provides you a concise overview of any integrations which are considered unhealthy, or in other words, integrations where the SamurAI platform has not seen events/data over specific time periods:

Figure 1: Example telemetry monitoring view

The fact that an integration is unhealthy doesn’t necessarily mean that there is a fault. For example, your integration may send telemetry intermittently. The SamurAI platform takes this into consideration through default status thresholds, however as environments may differ you may consider updating the Status Threholds to meet your requirements.

Telemetry Monitoring Notifications

By default the SamurAI platform will send email notifications to registered users if an integration is reported as Critical.

Notification settings can be fully customized to meet your requirements, refer to Notification Settings for more information.

Managing Integration Health

There are a few factors which could result in telemetry not being properly ingested. This article takes you through the main factors which could impact whether an integration is working or not, who is responsible for them, and how to address them.

In order for data to be ingested into the platform, the following main areas need to be functioning properly:

Platform is available: We are responsible for making sure that the SamurAI platform is available.
Data source configuration: Often the first place to check is that the source is correctly configured to send data. If your data source uses a Cloud Collector, you will also need to check that the Cloud Collector is functioning and healthy. Make sure that you have followed all of the configuration steps outlined in the configuration guide for the Integration.
Connectivity: Any data sources using Local Collectors are dependent on:
- required ports from the device to the Local Collector as outlined within the configuration guide
- internet connectivity between your premises and the SamurAI platform. Check that your internet connection is available and that firewalls are configured to allow traffic.
- the Local Collector article provides a detailed explanation of all of the ports that a Local Collector needs to have open in order to function correctly.
Local Collector: If your data source uses a Local Collector, you will need to ensure that the Local Collector is available. You will also need to ensure that the virtualization platform that hosts the Local Collector is healthy. For more information see the section on Local Collectors below.
Cloud Collector: If your data source uses a Cloud Collector, the health of your integration is also dependent on the Cloud Collector being operational. If your cloud collector monitors a cloud storage account, ensure that your integrated sources are sending and storing data to the storage account. If your data source is correctly configured but it remains unhealthy e.g reported status is Pending and/or Unknown, it is our responsibility to ensure the Cloud Collector is operational for you.

Local Collectors

If your integration is utilizing a Local Collector, first ensure it’s running as expected. If there is a problem with your Local Collector you should receive an email notification when the status is reported as Critical (by default) or per your configured user notification settings.

To receieve an email notification, notifications must be enabled for the Collector and user notifications settings configured. Refer to Collector Email Notifications and User Notifications Settings.

When you drill down into a Local Collector in the SamurAI Portal, you are provided a view which shows you the health of the Collector, together with all of the Integrations that are configured to use that Collector:

Figure 2: Example local collector view

For integrations that utilize a Cloud Collector you can jump directly to checking the Integration status.

Integration Status

Once you have confirmed that the Local Collector is Healthy (communicating with the SamurAI platform), check the Integration status. From the Collectors menu (applicable to both Local Collectors and Cloud Collector) you can view associated integrations to view their state of health. Alternatively, navigate to the Integrations page. Refer to Integrations Details and Status.

In both cases you will see a column called ‘Last Event Seen’. This column provides a timestamp of the last received event with the time format as [yyyy:mm:dd], [hh:mm:ss]) and in your timezone set per Time Zone.

The SamurAI platform monitors for ‘Last Event Seen’ within specific timeframes that relate directly to the status. There are default status thresholds, but they can also be fully customized, therefore if you are troubleshooting problems, check for any custom thresholds. Refer to Status Thresholds for more information.

If for some reason, the Integration is not healthy (e.g. not Green), then run through the Integration guide for your specific device and confirm there are no other controls blocking the traffic to the Local Collector or Cloud Collector.

If your Integration is of type Local or Cloud and is not healthy, then review the integration configuration to ensure it is correct and also ensure you followed the appropriate Integration guide for your device.

If you still have issues and please submit a ticket via the SamurAI Portal

Querying the detail

If you would like to go into more detail about the events from your log sources, you can make use of Advanced Query to analyze the events stored in the SamurAI platform. This will help you to answer questions like:

Is my log source generating logs intermittently?

By querying your log source over a period of time, the graphical representation of events will quickly show you time periods when your log source was not generating logs:

Figure 3: Example query

When did my log source last generate an event and what was that event?

You can easily find the last time when a log source generated an event. This will be the same as the “Last Event Seen” field for the Integration. For instance, the following query shows the last log generated in the last 7 days:

Figure 4: Example query

Is my log source configured to generate correctly formatted logs?

Sometimes a configuration error on your log source might result in your log source generating incorrectly formatted logs. By examining the raw log content you can check that your logs are correctly formatted. This will assist in correcting any configuration errors which may have resulted in incorrectly formatted logs being sent.

Is my log source sending the logs I need?

By checking the types of events generated, you can verify that you have configured the log source to send the logs you require, and that it is generating them. For instance, in this example, we are verifying that a device is generating DNS logs as expected:

Figure 5: Example query