Lorem ipsum dolor sit amet, conse ctetur adip elit, pellentesque turpis.

  • No products in the cart.

Image Alt

Real-Time Analytics using NiFi and Spark

  /    /  Real-Time Analytics using NiFi and Spark

Real-Time Analytics using NiFi and Spark

Big Data

The interactive virtual workshop is focussed on Building Data Flows, Performing Streaming Ingestion and Streaming Processing using NiFi and Spark Streaming. This program covers below aspects in NiFi and Spark –

  • Fundamentals, 
  • Limitations, 
  • Architectures, 
  • Hands-on Exercises

Spark API perspective below areas will be covered –

  • Micro-batch Processing 
  • Structured Streaming Processing

Hands-on exercises related to Spark will be in Python.

  • Application Developers
  • DevOps Engineers
  • Architects
  • System Engineers
  • Technical Managers
Introduction to Streaming
  • Streaming Overview
  • Streaming Use cases
  • Streaming Frameworks 
    • NiFi vs Kafka
    • Kafka Streams vs Spark Streaming
  • Challenges in Streaming 
  • Which one to choose when?
Introduction to NiFi
  • NiFi Overview
  • Limitations of NiFi
  • NiFi Terms
  • Hands-on: Setting up NiFi 
  • Demo: Working with NiFi User Interface
NiFi Architecture
  • Logical Architecture of NiFi
  • Physical Architecture of NiFi
  • Roles and Responsibilities of various components
  • Hands-on exercise 
Working with NiFi Flows
  • Anatomy of FlowFile
  • Key NiFi Processors
  • Configuring Queues 
  • NiFi Templates
  • Hands-on exercise
Delving Deeper in NiFi
  • NiFi Registry
  • NiFi Expression Language
  • Controller Service
  • Hands-on exercise
Integrating NiFi with Kafka
  • Role of Kafka
  • Kafka Terms
  • Kafka Processors
  • Hands-on exercise
Spark Streaming Overview
  • Spark Logical Architecture
  • Various APIs of Spark Streamings
  • DStream API vs Structured Streaming
  • Terms and Concepts
Streaming ETL using Spark
  • Processing streaming data using Spark 
  • Various Types of Windowing
  • Watermarking and Triggers Concept
  • Stateless Streaming
  • Hands-on exercise: Stateless Streaming
  • Stateful Streaming
  • Hands-on exercise: Stateful Streaming
  • Structured Streaming
  • Hands-on exercise: Structured Streaming
Integrations and Flows using Streaming Concepts
  • Structured Streaming Integrations 
  • Kafka Key Configurations
  • Hands-on: Integrating Kafka with Spark Streaming
  • Hands-on Integrate Spark Streaming with HBase
  • Hands-on: Build End to End Flow from Twitter to HBase
  • Basic Knowledge of Big Data Technologies as covered in Big Data Crash Course or equivalent knowledge
  • Basic knowledge of Python

Upon completion of this course, you should be able to:

  • Understand and Gain Hands-on exposure regarding 
    • Key Streaming Frameworks
    • Pros and Cons of NiFi, Spark and Kafka
    • Spark and NiFi Fundamentals & Architecture, 
    • Integration of NiFi and Spark Streaming via Apache Kafka
  • Spark Micro-batch processing 
  • Structured Streaming Processing

Course Information


3 Days

Mode of Delivery

Instructor led/Virtual

Reach out to us..Our representative will get back to you!

Post a Comment

Need Help? Chat with us
Please accept our privacy policy first to start a conversation.