Confluent Stream Processing using Apache Kafka® Streams & ksqlDB

Learn how to use Confluent ksqlDb to transform, enrich, filter, and aggregate streams of real-time data using SQL-like language.

Duration

3 Days

Level

Beginner to Intermediate Level

Design and Tailor this course

Official content and labs

Edit

During this hands-on course, you will learn to:

  • Identify common patterns and use cases for real-time stream processing
  • Understand the high level architecture of Apache Kafka® Streams
  • Write real-time applications with the Kafka Streams API to filter, transform, enrich, aggregate, and join data streams
  • Describe how ksqlDB combines the elastic, fault-tolerant, high-performance stream processing capabilities of Kafka Streams with the simplicity of a SQL-like syntax
  • Author ksqlDB queries that showcase its balance of power and simplicity
  • Test, secure, deploy, and monitor Kafka Streams applications and ksqlDB queries
Edit

This course is designed for application developers, architects, DevOps engineers, and data scientists who need to interact with Kafka clusters to create real-time applications to filter, transform, enrich, aggregate, and join data streams to discover anomalies, analyze behavior, or monitor complex systems.

Edit
  • Motivation and Use Cases for Real-Time Streaming
  • High Level Comparison of Kafka Streams and ksqlDB
  • Stream Processing Concepts
  • Kafka Streams’ Place in the Kafka Ecosystem
  • High Level Architecture Design
  • Kafka Streams Data Types
  • Get streams of data into and out of Kafka with Kafka Connect and REST Proxy
  • Maintain data formats and ensure compatibility with Schema Registry and Avro
  • Build real-time streaming applications with Confluent ksqlDB & Kafka Streams
  • Get streams of data into and out of Kafka with Kafka Connect and REST Proxy
  • Maintain data formats and ensure compatibility with Schema Registry and Avro
  • Build real-time streaming applications with Confluent ksqlDB & Kafka Streams
  • Unit Tests
  • Integration Tests
  • Stress Tests
  • End-to-end Tests
  • Sample Use Cases
  • End-to-end Examples
  • Interacting with ksqlDB
  • Data Manipulation
  • Aggregations
  • Testing
  • Parallelism
  • Elasticity
  • Fault tolerance
  • Capacity planning
  • Troubleshooting
  • ksqlDB-specific considerations
  • Security Overview
  • Access Control
  • Examples
  • ksqlDB-specific considerations
  • JMX
  • Confluent Control Center
  • ksqlDB-specific Considerations

Throughout the course, you will interact with hands-on lab exercises to reinforce stream processing concepts.

Exercises include:

  • Anatomy of a Kafka Streams Application
  • Joining Two Streams
  • Using the Kafka Streams Processor API
  • Testing a Kafka Streams Application
  • Using ksqlDB
  • Using the ksqlDB REST API
  • Scaling a Kafka Streams Application
  • Securing a Kafka Streams Application
  • Getting Metrics from a Kafka Streams Application
  • Using JConsole to monitor a Kafka Streams Application
  • Monitoring a Kafka Streams Application in Confluent Control Center
Edit

Attendees should be familiar with developing professional apps in Java (preferred), .NET, C#, Python, or another major programming language.
Additionally, students require a strong knowledge of Kafka architecture as well as knowledge of Kafka client application development, either through:

  • Prior experience, or
  • By taking the recommended prerequisites: Confluent Fundamentals for Apache Kafka® and Confluent Developer Skills for Building Apache Kafka®

Participants are required to provide a laptop computer with unobstructed internet access to fully participate in the class.

Connect

we'd love to have your feedback on your experience so far