Manage streams

This article covers creating and deleting streams and changing the configuration of a stream (Apache Kafka only) or deleting all records inside a stream (Apache Kafka only).

 

Are you looking for "Working with Views"?

The article about custom Views was moved to: Working with Views.

 

Creating a stream

After connecting to an environment, go to the data browser page and click on the top left triangle next to "Topics" or "Streams" if you are using Amazon Kinesis.

Create a new stream.

You will be prompted by the stream creation dialog.

Apache Kafka

If you are connected to an Apache Kafka cluster, you need to enter a name, the number of partitions and replications, and optionally additional configuration entries such as retention time or the retention policy.

Amazon Kinesis

If you are connected to Amazon Kinesis, you need to enter a name and the number of shards.

Click "Confirm" on the top right to create the stream.

Note: Make sure your replication factor is not higher than the maximum number of available brokers in your cluster and the topic name does not yet exist.

The topic creation dialog with configurations.

Verify that the topic with the requested number of partitions was created, by hovering over the topic in the browser.

Partition numbers are displayed on hovering over topic names.

 

Changing the configurations of a topic (Apache Kafka)

In the Stream Inspector select the stream on the left that you want to configure.

Select "Topic configuration" and add new configuration entries or change existing configuration entries by clicking on "Add configuration" or the pencil icon next to the existing configuration respectively. 

After you have made your changes, click on "Save changes".

 The topic configuration tab in the Stream Inspector view.

 

Wipe a topic (Apache Kafka)

You can delete all records of a topic.

In the Data Browser, right-click on the stream on the left side whose records you want to delete.

You will be prompted to confirm the deletion as this action cannot be undone. 

 The stream browser's context menu.

 

Deleting a stream

In the Data Browser, right-click on the stream on the left that you want to delete.

Select "Delete topic" for Apache Kafka or "Delete stream" for Amazon Kinesis from the context menu.

You will be prompted to confirm the deletion as this action cannot be undone. 

 The stream browser's context menu.

 

 

Right management

This section "Right management" only applies to KaDeck Web.

To complete the steps in this article, the user needs the following rights:

{
    "action": "TopicAccessAll",
    "effect": "Allow",
    "resource": "*:mybusiness.myteam.*"
  }

In the example above, the user has access to create, delete, or change the configuration of topics that start with the name mybusiness.myteam. on every cluster as indicated by the resource string. 

Alternatively, the following rights need to be granted:

{
    "action": "TopicAccessCreate",
    "effect": "Allow",
    "resource": "*:mybusiness.myteam.*"
},
{
    "action": "TopicAccessDelete",
    "effect": "Allow",
    "resource": "*:mybusiness.myteam.*"
},
{
    "action": "TopicAccessRead",
    "effect": "Allow",
    "resource": "*:mybusiness.myteam.*"
}