Example: Topics & Partitions
In this lesson, we'll study how our example is divided into topics and partitions.
Technical parameters of the partitions and topics
The topic order contains the order records. Docker Compose configures the Kafka Docker container based on the environment variable KAFKA_CREATE_TOPICS in the file docker-compose.yml in such a way as to create the topic order.
The topic order is divided into five partitions, as a greater number of
partitions allows for more concurrency. In the example scenario, it is
not important to have a high degree of concurrency. More partitions
require more file handles on the server and more memory on the client.
When a Kafka node fails, it might be necessary to choose a new leader
for each partition. This also takes longer when more
partitions exist.
This argues for a lower number of partitions as used in the example in order to save resources. The number of partitions in a topic can still be changed after creating a topic. ...