Course Outline
NiFi and Data Flow Foundations
- Contrasting data in motion versus data at rest: key concepts and associated challenges
- NiFi architecture overview: cores, flow controller, provenance, and bulletin board
- Core components: processors, connections, controllers, and provenance tracking
Big Data Ecosystem Context and Integration
- NiFi's role within Big Data environments (including Hadoop, Kafka, and cloud storage)
- Introduction to HDFS, MapReduce, and contemporary alternatives
- Practical applications: stream ingestion, log transmission, and event pipelines
Deployment, Configuration, and Cluster Establishment
- Setting up NiFi on single-node systems and in cluster modes
- Cluster setup: defining node roles, Zookeeper integration, and load balancing
- Managing NiFi deployments via Ansible, Docker, or Helm
Dataflow Design and Management
- Techniques for routing, filtering, splitting, and merging flows
- Configuring processors (such as InvokeHTTP, QueryRecord, PutDatabaseRecord, etc.)
- Managing schemas, enrichment, and transformation tasks
- Strategies for error handling, retry mechanisms, and backpressure management
Integration Scenarios
- Linking with databases, messaging platforms, and REST APIs
- Directing streams to analytics platforms like Kafka, Elasticsearch, or cloud storage
- Connecting with tools such as Splunk, Prometheus, or logging pipelines
Monitoring, Recovery, and Provenance
- Utilizing the NiFi UI, performance metrics, and provenance visualizer
- Establishing autonomous recovery protocols and graceful failure handling
- Implementing backups, flow versioning, and change management processes
Performance Tuning and Optimization
- Adjusting JVM, heap, thread pools, and clustering parameters
- Refining flow design to mitigate bottlenecks
- Managing resource isolation, flow prioritization, and throughput control
Best Practices and Governance
- Documentation standards, naming conventions, and modular design principles
- Security measures: TLS, authentication, access control, and data encryption
- Governance controls: versioning, role-based access, and audit trails
Troubleshooting and Incident Response
- Addressing common issues: deadlocks, memory leaks, and processor failures
- Analyzing logs, diagnosing errors, and investigating root causes
- Executing recovery strategies and flow rollbacks
Practical Lab: Implementing a Realistic Data Pipeline
- Constructing an end-to-end flow covering ingestion, transformation, and delivery
- Applying error handling, backpressure controls, and scaling techniques
- Conducting performance testing and pipeline tuning
Conclusion and Future Directions
Requirements
- Proficiency with the Linux command line
- Foundational knowledge of networking and data systems
- Familiarity with data streaming or ETL concepts
Intended Audience
- System administrators
- Data engineers
- Software developers
- DevOps practitioners
Testimonials (7)
Hands on exercises. Class should have been 5 days, but the 3 days helped to clear up a lot of questions that I had from working with NiFi already
James - BHG Financial
Course - Apache NiFi for Administrators
I thought the trainer's pace was good. He left no student behind with his approach. He was very supportive with us NEWBIES that may not have had a System Administrator or Infrastructure role during our career or resurrected those skills from a prior period of our career.
Pamdrea Ivory - BHG Financial
Course - Apache NiFi for Administrators
I like the hands on section. It helped me to better retain information by completing the provided exercises. Also, the trainer's ability to engage with the entire class made me feel comfortable to ask questions on things I was not sure about.
Leila - BHG Financial
Course - Apache NiFi for Administrators
Use-cases, examples for building NiFi dataflows. We worked on troubleshooting common problems and gotchas.
Nelson - BHG Financial
Course - Apache NiFi for Administrators
I loved the structure. We dove into the basics of Nifi, concepts, use cases, etc. on day 1. On day 2, we got to put Day 1 knowledge into practice by building out flows to meet scenario requirements. Day 3 we got to see Nifi Registry and version control, mulit-tenancy, and go over Q&A.
Adam - BHG Financial
Course - Apache NiFi for Administrators
I like how he was able to elaborate about Nifi and how powerful it is. You can basically use it for any infrastructure and use many different computer languages. Also i was glad we were able to fix the Nifi cert renewal issue we were having with the Truststore.
Joachim Martin - BHG Financial
Course - Apache NiFi for Administrators
general knowledge and the possibilities that the training offered in terms on the tool.
