Stephane Derosiaux

Camp Counselor

Apache Kafka Demystified: Breaking Down the Basics

Event Logo

Tuesday, July 30, 2024 - 7:30 PM UTC, for 1 hour.

Regular, 60 minute presentation

Room: African 60

kafka
data mesh
streaming
self-service
event-driven

Apache Kafka is essential for organizations of all sizes to work with real-time data and be competitive. It is loved by developers, QA teams, and architects, and sometimes hated because of its complexity. Kafka enables efficient data exchange between teams at scale through its distributed streaming platform and extensive tooling, forming the backbone of many products and data systems. We'll explore why Apache Kafka is both powerful and simple yet complicated. We'll look at its key features like topics, partitions, and consumer groups using Conduktor to show how it operates and why it's designed like this. We'll also create stream processing applications to showcase its real-time capabilities and how it connects with traditional systems like databases. Additionally, we'll highlight some common issues in Kafka-based applications, typical of distributed systems, and how to tackle them.

Prerequisites

Folks will benefit most from this talk if they have experience working with real-time data. We'll be discussing distributed system concepts and stream processing. Experience with any message brokers would be useful.

Take Aways

  • Learn about Kafka and how to develop applications with it
  • Build and connect applications via Kafka in real-time
  • Watch out for tricky parts in Kafka
favorited by:
Micah Silverman Tristan Dalgety Allan Wick Tadd Eells Dustin Ewers Jesse Dahir-Kanehl Paul Ross Chris Lane Jones Spenser Austin Edward Lichtman Jonathan Renier-Wigg Nick Heidke Stephane Derosiaux Steve Riebe Kenzie Johnson Yuhe Yuan Sam Patterson William Schaeffer Stephen Rimbakusumo Craig Jeffords Tim Kempster Caleb Soper