Apache Kafka data input configuration fields

reference brazil Updated Sep 10, 2026 4 min read hla-data-input-kafka-ref.md
Raw file servicenow.com

Description of the fields on the Apache Kafka data input configuration form.

Field

Description

Name

Name of the new data input. This field is required.

Description

Optional description of the data input.

Execute on

Option to select whether to use a specific MID Server or a specific MID Server cluster. This field is required.

MID

(Only when the Execute on field is set to Specific MID Server)

The MID Server to which log data from Apache Kafka is pulled. This field is required.Note:

  • The default maximum number of data inputs streaming logs to a single MID Server is 10. You can modify this number in the MID Server properties.
  • If log ingestion is not enabled for the selected MID Server, Health Log Analytics enables it automatically.

MID Server Cluster

(Only when Execute on is set to Specific MID Server cluster.)

The MID Server cluster to which the log data is pulled. This field is required.The data input runs on a single MID Server in the cluster until that MID Server fails. The system then moves all the data input tasks to the next available MID Server in the cluster according to the configured order.

Note:

  • Health Log Analytics supports only failover MID Server clusters. In these clusters, multiple MID Servers are grouped together for failover protection. When selecting a cluster from the data input or integration form, the MID Server clusters list displays only failover clusters.
  • Log ingestion must be enabled for each MID Server in the cluster. If log ingestion is not enabled for the active MID Server, Health Log Analytics enables it automatically.
  • The default maximum number of data inputs or integrations streaming logs to a single MID Server is 10. A cluster passes capacity validation if it contains at least one MID Server with fewer than 10 data inputs or integrations running on it. This is true even when that MID Server is down.

For more information about MID Server clusters, see Configure a MID Server cluster.

Service instance

The service instance to which to bind the log data. This field is required.Selecting the service instance enables HLA to correlate CIs and service instances in the CMDB for root cause analysis.

Note: If no appropriate service instance exists, create one, add CIs to it, and set its status to Operational. For more information, see Create a Service instance (application service).

Field

Description

Status

Status of the data input. This field is read-only.

Transport

Protocol used to stream the log data. This field is read-only.This data input uses Apache Kafka to stream log data to your instance.

Sources count

The number of log sources this data input has created. This field is read-only.

Disabled since

The time when the data input stopped or failed. This field is read-only.

MID last receiving time

The time when the last log streamed in the data input.

Error message

The streaming error.This field is populated automatically. It displays only when a streaming error has occurred.

Advanced

Option to display the Advanced tab for advanced configuration.

Field

Description

Example

From

Starting date and time for reading the data. Data older than this date and time is not read. This field is required.Note: Setting this value to a past date might require the system to read large amounts of data, causing congestion.

Now -1 week

Field

Description

Example

Kafka node names

A comma-separated list in the format HOST:PORT,HOST:PORT. The list does not have to include all the Apache Kafka Cluster servers. This field is required.

123.4.5.6:9092,123.3.4.5:9093

Topics

A comma-separated list of topics to which the data input must subscribe. This field is required.

FirstTopic,SecondTopic,ThirdTopic

Kafka credentials

Reference to the Apache Kafka credentials.You can display the Kafka SSL credentials form by selecting the Search icon ([Omitted image "List_SearchIcon.png"]) and opening a record. The form enables you to choose the security protocol used for authenticating with Apache Kafka from the following options:

  • SSL - SSL channel.
  • SASL_SSL - SASL authenticated, SSL channel.
  • SASL_PLAINTEXT - SASL authenticated, non-encrypted channel.

You can also create a record, by selecting New and completing the credentials form for the selected protocol. For a description of the fields on the form, see Kafka SSL credentials fields.

None

Group ID

The name of the Apache Kafka Consumer Group.

logs

|Field|Description|Default value| |-----|-----------|-------------| |Timeout|The time, in milliseconds, spent waiting in the poll if data is not available in the topics.|500| |Node discovery timeout|The time, in milliseconds, before node discovery times out.|30| |Default timezone|The default time zone used when a log doesn't include time zone information.|GMT| |Sub sample drop ratio|The number of events to batch together, out of which one will be discarded. This setting is used to reduce the number of fetched events.|-1| |Sub sample receive ratio|The number of events to batch together, out of which all but one will be discarded. This setting is used to decrease the number of received events.|-1| |Max length in bytes|The maximum length, in bytes, of events.|32766| |Character encoding|The character encoding for this data input. This field is read-only.|UTF-8| |Drop if queue is full|Option to discard logs if many processes are waiting in the queue to access the MID Server.|Clear|

Parent Topic:Data input configuration fields

Search documentation
Search document titles and file paths on the brazil branch