Configure an Extract Transform Load \(ETL\) consumer

task brazil Updated Sep 10, 2026 2 min read configure-etl-consumer.md
Raw file servicenow.com

Import and process data from your Kafka environment using your existing Robust Transform Engine (RTE) configurations.

Before you begin

About this task

To configure a consumer, you need to create two records.

  1. The consumer record, which specifies how to import and process data.
  2. A record for the Kafka stream, which defines the stream of data to your consumer.

This task covers the consumer creation. For instructions on creating a Kafka stream, see Create a Kafka stream.

Procedure

  1. Navigate to All > IntegrationHub > Consumers > ETL Consumer.

  2. Select New.

  3. In the form, fill in the fields.

Field

Description

Name

Name of the ETL consumer.

Robust import set transformer

Name of the robust import set transformer to use to process data.

Delivery guarantee

If there's a node failure, option to specify the delivery guarantee for incoming messages. Select one of the following.- No lost but duplicates: All messages are delivered at least once. Some messages might be delivered more than once.

  • Once or not at all: A message isn’t delivered more than once. Some messages might not be delivered at all.

Serialization format

The serialization format for the message. Select one of the following. - Plain Text: Select this option for any plain-text messages. This is the default format.

  • Encoded: Select this option for messages in an Apache Avro format. Converting plain-text messages to an Avro format requires a schema. Select the schema registry in the Schema registry field. For more information on schemas, see Schema management in Stream Connect.

Data in single column

Option to have the import set table store data in single column.

Column mapping

Option to specify whether the message's JSON key maps the data to the column's name or label in the import set table. Select one of the following.- Label

  • Column name This field appears only when the Data in a single column check box is cleared.

Application

Application scope for the ETL Consumer.

Schema registry

Registry for the selected schema. Select one of the following.

  • Standalone Schema Registry
  • Confluent Schema Registry This field appears only when the Serialization format is set to Encoded.

For the Confluent Schema Registry, if the received message's schema ID isn't in the schema table, the system imports the schema dynamically, using the configured REST connection.

4. Select **Save**.

What to do next

Create a Kafka stream for this consumer. After the stream is activated, you can start receiving messages from your Kafka environment.

Parent Topic:Using Stream Connect for Apache Kafka

Search documentation
Search document titles and file paths on the brazil branch