AI Features

0% completed

All LessonsFree Lessons (5)

Basics

Introduction Characteristics of Distributed Systems Messaging Patterns Commit Log Components of Kafka Partitions Partition Rebalancer

Kafka Producer

Sending Messages

Producer Configurations

Producer Serialization

Kafka Consumer

Consumer and Consumer Groups Kafka Consumer Consumer Configurations Commits and Offsets Offset Commit Configuration Handling Rebalances Stopping a Consumer Consumer Deserializer Running a Single Consumer

Kafka Internals

Replication Kafka Controller Request Processing Partition Allocation Data Storage Reliability in Kafka Reliable Producers and Consumers

Conclusion

Kafka vs Other Messaging Systems

Appendix

Zookeeper: Intro Zookeeper: Example Zookeeper: Practical

Reference: Replication

Introduction Single Leader Replication Asynchronous VS Synchronous Replication Followers in Log Replication Log Replication Issues in Single Leader Replication Multi Leader Replication Issues in Multi Leader Replication Multi-Leader Topologies Leaderless Replication R + W > N Quorum Variations Concurrent Writes Video Streaming Queue a Concurrent Writes Example

Reference: Partitioning

Introduction Partitioning Schemes Secondary Indexes Number of Partitions

Reference: Transactions

Introduction to Transactions More on Transactions Isolation Levels Read Skew and Snapshot Isolation Concurrent Writes and Lost Updates Write Skew Serializability Two Phase Locking Serializable Snapshot Isolation

Reference: Issues in Distributed Systems

Introduction Networking Agreeing On Time Working with Time Issues

Building Scalable Data Pipelines with Kafka

Producer

This lesson explains how Kafka producers write messages to topic partitions.

We'll cover the following...

Write workflow
Example

Producers write messages to Kafka that can then be consumed by readers. We’ll take a detailed look at how messages get written to Kafka in this lesson.

Write workflow

The steps involved are:

An object of class ...

Ask