The Kafka profile configuration contains two tabs: Connectivity and Advanced. The Connectivity tab is displayed by default when creating or opening a Kafka profile, and it contains the following settings: Topic Enter the Kafka topic that you want to use for your configuration. For information on how to create a Kafka topic, refer to /wiki/spaces/MD/pages/3214953. This field supports parameterization using ${} syntax, see Profiles(3.0) for more information on how parameterization works. Use parameterized broker list Enable this checkbox if you like to set Brokers as a parameter instead. When enabled you need to set parameter name using ${} syntax, see Profiles(3.0) for more information on how parameterization works. Host If you are using external Kafka, enter the host name for Zookeeper. Port If you are using external Kafka, enter the port for Zookeeper. Set this id to an identity of your choice, to enable load sharing between several Kafka collectors. Use this in combination with This field supports parameterization using ${} syntax, see Profiles(3.0) for more information on how parameterization works. Security Profile Connect the Security (3.0), in the case you have defined it for your Kafka server. In the Advanced tab you can configure properties for optimizing the performance of the Kafka Producer and Consumer. The Advanced tab contains two tabs: Producer and Consumer. In the Producer tab, you can configure the properties of the Kafka forwarding agent. The property When running in Acknowledged execution mode, the property For information on how to configure the properties for SSL and Kerberos, please refer to https://www.cloudera.com/documentation/kafka/latest/topics/kafka_security.html. This field supports parameterization using ${} syntax. For more information on parameterization see Profiles(3.0). In the Consumer tab, you can configure the properties of the Kafka collection agent. For further information about the properties, see the text in the Advanced consumer properties field, or refer to https://kafka.apache.org/25/documentation.html#consumerconfigs. This field supports parameterization using ${} syntax. For more information on parameterization see Profiles(3.0).The Kafka profile enables you to configure which topic and which embedded service key to use, and it is loaded when you start a workflow that depends on it. Changes to the profile become effective when you restart the workflow.
Connectivity tab
Setting Description Brokers A Broker is a node in a Kafka cluster. If you are using external Kafka, you must add Kafka Brokers. Use the Add button to enter the addresses of the Kafka Brokers that you want to connect to. Group id enable.auto.commit=true, in the Consumer and Advanced tab.Advanced tab
Producer tab
producer.abortunknown=true sets the agent to abort if the broker replies with Unknown topic or partition. For further information on the other properties, see the text in the Advanced producer properties field, or refer to https://kafka.apache.org/25/documentation.html#producerconfigs.producer.full.response determines if the data sent to the kafka log is also included in the response UDR. The value is set to true by default. Setting the value to false reduces the memory footprint.Consumer tab
Manage space
Manage content
Integrations
App links
Add Comment