Publisher : DataShark Academy
Course Length : 37 mins
Course Language : English
ABOUT THIS COURSE
In this beginner course you will learn few core concepts about Apache Kafka which will provide you a good start. After completing this course, you can take full course from DataShark Academy to become Apache Kafka Guru. The link to full course will be in the last lecture. Once you complete full course, You will be able to build Kafka applications for any real world use cases.
In more detailed course - Apache Kafka Guru, you will get following:
Learn Apache Kafka with plenty of hands on practicals !!!
Just in few minutes you will be on the route to be an Apache Kafka hero. You will learn in this course how 1000's of companies have built their business solutions (ex. Netflix) around Apache Kafka.
This comprehensive Kafka tutorial starts with building foundation about all major concepts in Apache Kafka. This part of the course is vital for understanding how Kafka really works. This section will help you with hands-on exercises later in this course.
In the later part of this Kafka tutorial, we will have lots of hands-on exercises covering from basic to advanced topics of Kafka. You will learn how to set up Apache Kafka on your personal computer (Mac/Linux or Windows PC). Then create Kafka topics (where actual data is stored inside Kafka) & perform various operations on your topics. Later you will setup Kafka producers and Kafka consumers to send and receive data.
In more advance topics you will learn how to create your own Kafka Producers and Kafka Consumers.
WHY APACHE KAFKA
Apache Kafka is one of the most promising data processing system available today. Apache Kafka is an open source distributed streaming platform which can handle 100s of billions of events in a day.
It is giving tough competition to other streaming applications such as Apache Spark & Flink.
Apache Kafka is easy to setup and learn. It enables you to connect multiple systems to share the data in 100s of different possible combinations. For instance, using Apache Kafka, you can connect a legacy Relation Database System (RDBMS) with Apache Hadoop's Distributed File System, or with ElasticSearch in ELK stack. You can also capture data from one system and send it to AWS S3 or AWS Lambda functions. The possibilities are endless.
That's why every data engineer and data architect must learn Apache Kafka today.
In complete course, you will also learn how to create your own custom Kafka Producers and Kafka Consumers using Java code in easy to understand manner. These exercises will greatly boost your confidence to build any custom producers and consumers for real world problems.
Here are some of topics that we will cover in full course:
Section 1 – APACHE KAFKA & IT’S FEATURES
In this section of the course, we will start with building the knowledge about various key concepts of Apache Kafka. This will be important section, so make sure you watch all videos before moving to the next section of the course. Here are some of the chapters covered in this section:
What is Apache Kafka?
How Apache Kafka provides High Level of Scalability?
Kafka’s In-built Fault Tolerance
Apache Kafka is damn Popular
Section 2 – WHY APACHE KAFKA?
In this part of the course, we will discuss why we really need Apache Kafka and what kind of real world problem does it solve? Here are some of the chapters covered in this section:
Why we need Apache Kafka?
Decoupling of Systems
Section 3 – APACHE KAFKA ARCHITECTURE
It’s time to dig deeper into how Apache Kafka really works and we will do this by understanding Kafka’s architecture.
Apache Kafka Architecture
How Netflix is using Kafka
Section 4 – ANATOMY OF APACHE KAFKA
In this section of the course, we will look at various components that make Apache Kafka what it is today. This is an important section of this course. It will help you really build the core understanding about each individual component of a Kafka application.
What is Kafka Broker
Kafka Topic Explained
Learn about Kafka Topic Partitions
What are Kafka Offsets
How Replication Works in Kafka
Who is the leader
Let’s talk about Zookeeper
Putting everything together
Section 5 – DESIGNING APACHE KAFKA APPLICATIONS
In this section, we will talk about what to consider while designing Apache Kafka applications.
Section 6 – EXPERT LEVEL HANDS-ON
Let’s use the knowledge and get our hands dirty by working on some real exercises.
Setup Apache Kafka on Local Computer (Mac & Linux users)
Setup Apache Kafka on Local Computer (Windows PC users)
How to connect to HDP Sandbox terminal (Windows PC users)
Let’s fire up a local cluster (Windows PC users)
It’s time to create some Kafka Topics
What Kafka topics you got?
Peak inside a Kafka topic
How to delete a Kafka topic
Running a Kafka Producer
Running a Kafka Consumer
What to do when a Kafka Producer or Kafka Consumer goes down in production
Section 7 – ADVANCE LEVEL HANDS-ON
In this part of the course, you will work on more advanced subjects.
How to create your own Kafka Producer (Java Code)
How to create your own Kafka Consumer (Java Code)
Integrating Kafka with Spark Streaming (Scala Code)
Running your Spark Data aggregator
Section 8 – BONUS LECTURES
Trick #1 – Generate data for your Kafka Console Producer
Trick #2 – Simulate a Real Time Streaming Data Source
This course is taught by professionals with extensive experience in handling big data applications for top fortune 100 companies of the world.
Your instructors have managed to create data pipelines for extracting, transforming & processing over 100's of Terabytes of data in a day for their clients providing data analytics for user services.
At DataShark Academy, we provide accelerated learning programs taught by professionals with years of expertise in Big Data technologies and working with 10s of clients.
You will learn plenty using our unique approach that focuses on maximum results in the shortest possible time.