Big Data Learning Resources

Complete Plan to learn Big Data Step by Step (All Free resources Included) by Sumit Sir.

1. Learn SQL Basics – https://lnkd.in/g9NEJMVE
SQL will be used at a lot of places – Hive/Spark SQL/RDBMS queries
Joins & windowing functions are very important

2. Learn Programming/Python for Data Engineering – https://lnkd.in/gr6fFPdU
Learn Python to an extent required for Data Engineers.

3. Learn the Fundamentals of Big Data – https://lnkd.in/gXBtMQvU
Understand what is Big Data, Idea about hadoop ecosystem, Understand what is ingestion, processing, serving and other layers.

4. Learn Azure Datalake Storage – https://lnkd.in/gY8q_HyN
you should know about distributed storage like HDFS & object store like your blob storage. Understand about Datalake ADLS gen2 / Amazon S3 for your storage needs

5. Distributed Processing with Pyspark – https://lnkd.in/g3DdahwX
Start with RDD but your major focus should be on Structured API’s like Dataframes and spark sql, Cover performance tuning extensively

6. Learn Azure Databricks – https://lnkd.in/gh4cMXmX
Databricks is adopted by lot of companies these days and things like Delta Lake to provide ACID guarantees makes it quite useful.
Learn Lakehouse architecture, delta lake, medallian architecture, unity catalog, autoloader, DLT

7. Learn Azure DataFactory – https://lnkd.in/gHM6ap6M
Learn Datafactory for your Ingestion and Scheduling needs

8. Prepare for Interviews – https://lnkd.in/g2a4E3Dq
Learn Data Modelling/System Design/Deployment & Prepare for Managerial round questions

Now you can start giving interviews… but do not stop, you should still focus to learn more on Streaming part

9. Learn Structured Streaming with Kafka – https://lnkd.in/g4awz6hn
Its a super solid combination, Try to handle all the scenario like what if the streaming pipeline fail, how to handle failures etc..

10. Learn more Cloud services – https://lnkd.in/g6ikJqdW
you can learn other cloud services in AWS and Azure this should be helpful when you are designing a solution architecture

Additionally –
1. Learn DSA – some of the top companies will definitely ask questions on Data Structures.

2. Do few projects to gain the confidence.

This is a rock solid learning Plan.