Apache Kafka
Categories:
Big Data
Reviews:
Apache Kafka is an open-source stream-processing software platform developed by LinkedIn and donated to the Apache Software Foundation, written in Scala and Java. Apache Kafka is a community distributed event streaming platform capable of handling trillions of events a day. Initially conceived as a messaging queue, Kafka is based on an abstraction of a distributed commit log.
The program is focussed on Data Ingestion using Kafka. This program covers Kafka Fundamentals, Architecture, API, Administration, Kafka Connect and Kafka Streams. The course will have lot of hands-on exercises on Confluent Kafka platform. Hands-on exercises will be either in Java or Scala.
-
- Understand Apache Kafka Ecosystem, Architecture, Core Concepts and Operations
- Create your Producers and Consumers in Java to interact with Kafka
- Master Concepts such as Topics, Partitions, Brokers, Producers, Consumers
The intended audience for this course:
- Application Developers
- DevOps Engineers
- Architects
- System Engineers
- Technical Managers
Introduction to Apache Kafka
- Kafka Overview
- Salient Features of Kafka
- Kafka Use cases
- Comparing Kafka with other Key tools
Kafka Fundamentals & Internals
- Logical Architecture of Kafka
- Physical Architecture of Kafka
- Roles and Responsibilities of various components
- Replication mechanism
- Internal data storage on Brokers
- Log compaction
- Message Delivery Semantic
- Key Terminologies
Zookeeper
- Role of Zookeeper
- Zookeeper Basic Operations
- Apache Kafka – Zookeeper Role
Kafka Integrations
- End to End Data Pipeline using Kafka
- Kafka Connect
- Hands-on Exercises
Kafka API
- Overview
- Producer API
- Consumer API
- Hands-on Exercises
Schema Management in Kafka
- An Introduction to Avro
- Avro Schemas
- Using the Schema Registry
Administration
- Installation
- Configuration
- Monitoring
- Administering Kafka
- Migration aspects
- Backup and Disaster Recovery
- Best practices
Kafka Streams
- Overview
- Understand What is Spark
- Describe RDDs
- Explain Spark Components
- Integrate Kafka with Spark
- Kafka Streams
- Spark Integration Approaches
- Integrating Kafka with Spark Streaming
- Hands-on Exercise
Participants should preferably have basic knowledge of a programming language like Java and basic knowledge of Unix commands.