Course Code

sparkcloud

Duration

21 hours (usually 3 days including breaks)

Requirements

Programing skills (preferably python, scala)

SQL basics

Overview

Apache Spark's learning curve is slowly increasing at the begining, it needs a lot of effort to get the first return. This course aims to jump through the first tough part. After taking this course the participants will understand the basics of Apache Spark , they will clearly differentiate RDD from DataFrame, they will learn Python and Scala API, they will understand executors and tasks, etc. Also following the best practices, this course strongly focuses on cloud deployment, Databricks and AWS. The students will also understand the differences between AWS EMR and AWS Glue, one of the lastest Spark service of AWS.

AUDIENCE:

Data Engineer, DevOps, Data Scientist

Course Outline

Introduction:

Apache Spark in Hadoop Ecosystem
Short intro for python, scala

Basics (theory):

Architecture
RDD
Transformation and Actions
Stage, Task, Dependencies

Using Databricks environment understand the basics (hands-on workshop):

Exercises using RDD API
Basic action and transformation functions
PairRDD
Join
Caching strategies
Exercises using DataFrame API
SparkSQL
DataFrame: select, filter, group, sort
UDF (User Defined Function)
Looking into DataSet API
Streaming

Using AWS environment understand the deployment (hands-on workshop):

Basics of AWS Glue
Understand differencies between AWS EMR and AWS Glue
Example jobs on both environment
Understand pros and cons

Extra:

Introduction to Apache Airflow orchestration

Testimonials

★★★★★

Richard is very calm and methodical, with an analytic insight - exactly the qualities needed to present this sort of course.

Kieran Mac Kenna

Course:Spark for Developers

We know a lot more about the whole environment.

John Kidd

Course:Spark for Developers

The trainer made the class interesting and entertaining which helps quite a bit with all day training.

Ryan Speelman

Course:Spark for Developers

Ernesto did a great job explaining the high level concepts of using Spark and its various modules.

Michael Nemerouf

Course:Spark for Developers

Richard was very willing to digress when we wanted to ask semi-related questions about things not on the syllabus. Explanations were clear and he was up front about caveats in any advice he gave us.

ARM Limited

Course:Spark for Developers

The VM I liked very much The Teacher was very knowledgeable regarding the topic as well as other topics, he was very nice and friendly I liked the facility in Dubai.

Safar Alqahtani - Elm Information Security

Course:Big Data Analytics in Health

Small group (4 trainees) and we could progress together. Also the trainer could so help everybody.

ICE International Copyright Enterprise Germany GmbH

Course:Spark for Developers

Ajay was very friendly, helpful and also knowledgable about the topic he was discussing.

Biniam Guulay - ICE International Copyright Enterprise Germany GmbH

Course:Spark for Developers

The lab exercises. Applying the theory from the first day in subsequent days.

Dell

Course:A Practical Introduction to Stream Processing

I think the trainer had an excellent style of combining humor and real life stories to make the subjects at hand very approachable. I would highly recommend this professor in the future.

Course:Spark for Developers

The trainer was passionate and well-known what he said I appreciate his help and answers all our questions and suggested cases.

Course:A Practical Introduction to Stream Processing