Legacy Kafka Profile

The Kafka profile enables you to configure which topic and which embedded service key to use, and it is loaded when you start a workflow that depends on it. Changes to the profile become effective when you restart the workflow.

The Kafka profile configuration contains two tabs: Connectivity and Advanced.

Connectivity tab

The Connectivity tab is displayed by default when creating or opening a Kafka profile, and it contains the following settings:

Setting	Description

Setting	Description
Topic	Enter the Kafka topic that you want to use for your configuration. This field supports parameterization using ${} syntax, see Appendix 1 - Profiles for more information on how parameterization works.
Use parameterized broker list	Enable this checkbox if you want to set Brokers as a parameter instead. When enabled you need to set the parameter name using ${} syntax, see Appendix 1 - Profiles for more information on how parameterization works.
Brokers	A Broker is a node in a Kafka cluster. If you are using external Kafka, you must add Kafka Brokers. Use the Add button to enter the addresses of the Kafka Brokers you want to connect to.
Host	If you are using external Kafka, enter the hostname for Zookeeper.
Port	If you are using external Kafka, enter the port for Zookeeper.
Group id	Set this id to an identity of your choice, to enable load sharing between several Kafka collectors. Use this in combination with `enable.auto.commit=true`, in the Consumer and Advanced tab. This field supports parameterization using ${} syntax, see Appendix 1 - Profiles for more information on how parameterization works.
Security Profile	Select a Security Profile profile if you have defined it for your Kafka server.

Advanced tab

In the Advanced tab you can configure properties for optimizing the performance of the Kafka Producer and Consumer. The Advanced tab contains two tabs: Producer and Consumer.

Producer tab

In the Producer tab, you can configure the properties of the Kafka forwarding agent.

The property producer.abortunknown=true sets the agent to abort if the broker replies with Unknown topic or partition. For further information on the other properties, see the text in the Advanced producer properties field, or refer to https://kafka.apache.org/25/documentation.html#producerconfigs .

When running in Acknowledged execution mode, the property producer.full.response determines if the data sent to the Kafka log is also included in the response UDR. The value is set to true by default. Setting the value to false reduces the memory footprint.

For information on how to configure the properties for SSL and Kerberos, please refer to Configuring Apache Kafka Security | 4.1.x | Cloudera Documentation .

Caution!

If you make any changes to the security configuration of the Kafka Producer, any topics used must be recreated before they can be used.

This field supports parameterization using ${} syntax. For more information on parameterization see Appendix 1 - Profiles.

Consumer tab

In the Consumer tab, you can configure the properties of the Kafka collection agent.

For further information about the properties, see the text in the Advanced Consumer Properties field, or refer to https://kafka.apache.org/25/documentation.html#consumerconfigs .

This field supports parameterization using ${} syntax. For more information on parameterization see Appendix 1 - Profiles.

Note!

The sasl.jaas.config client property has been added to Advance Producer Properties and Advance Consumer Properties in the Kafka profile.

This new property is used to configure SASL authentication directly in the client's properties instead of using a JAAS file. This simplification lets you run multiple clients in the same JVM by using different sets of credentials, which is not possible with a JAAS file.

You can still use the existing java.security.auth.login.config system property which points to a JAAS file. However, this option allows only one set of user credentials for all client connections from a JVM. This means that MediationZone users won’t be able to run multiple Kafka workflows against different Kafka brokers on the same EC/ECSA.

When both the JAAS configuration system property (java.security.auth.login.config) and client property (sasl.jaas.config) are specified, the client property will be used.