This article describes how to set up and connect to the embedded cluster with KaDeck.
What is the embedded cluster functionality?
With KaDeck 1.4 "embedded cluster" was introduced. This new functionality for fast local testing allows you to start up an Apache Kafka broker (and Zookeeper) directly from KaDeck by keeping the flexibility to manually change all broker settings (Kafka and Zookeeper) and to choose the Apache Kafka version you want.
Head over to the Apache Kafka project page and download the binary version of Apache Kafka. Un-tar the downloaded folder and move it to a location that is accessible for KaDeck.
For Windows users: as there are some constrains on path lengths on Windows which can result in strange behavior and weird error messages, we recommend placing the folders content close to the root of your hard drive (e.g. C:/kafka/ so that the path to /bin is C:/kafka/bin).
Next start KaDeck and enter the path to your extracted folders content (e.g. C:/kafka/) and click "Start".
Fig. 1 Embedded cluster card in online state.
Click on "Start" and wait several seconds. The starting process can take a while depending on your machine. This does not automatically connect you to the local cluster.
Once your cluster is running, you can connect using a new server configuration with the local broker address (usually 127.0.0.1:9092).
If an error occurred, check the log files in your folder (e.g. C:/kafka/zookeeper.out).
Note: Clicking on "Start" does kill any previous Kafka broker and Zookeeper instances first, deletes the data dir and log dir and the starts both servers normally.
There are no topics (or data) after restarting my embedded cluster.
This is normal as KaDeck does delete all information from previous sessions once the embedded cluster is started. This guarantees that you are starting with a clean environment every time.
I have closed KaDeck without stopping the cluster.
Simply open KaDeck again and click on "Start" on the embedded cluster card. This will stop any instances (if existent) and start the server again. You can then stop the server normally by clicking "Stop".
I want to change the Kafka or Zookeeper settings
You can do so by editing the server.properties and zookeeper.properties inside the /config folder.