SET OF 82 QUESTIONS 1. How is Apache Spark different from MapReduce? Apache SparkMapReduceSpark processes data in batches as well as in real-timeMapReduce processes data in batches onlySpark runs almost 100 times faster than Hadoop Map ReduceHadoop MapReduce is slower when it comes to large sc processingSpark stores data in the RAM i.e. in-memory. So,... Continue Reading →
Google Cloud Compute Engine vs App Engine
Google Cloud Platform provides a wide range of computing services that target broad categories of user needs. The Google Cloud Platform provides mainly 6 types of compute options: – App Engine Compute Engine Kubernetes Engine Cloud Functions Cloud Run VMware Engine Now let’s talk about some of these services in brief. Compute Engine The Compute... Continue Reading →
Google Cloud Platform Services Summary
The complete list of services that form Google Cloud Platform is shown below. While Google offers many other services and APIs, only the services below are covered by the Google Cloud Platform terms of service, service level agreements (if applicable), and support offerings. Offerings identified below as Software or Premium Software are not Services under... Continue Reading →
Google Cloud GCloud Commands Cheat Sheet
Google Cloud Config PURPOSECOMMANDList projectsgcloud config list, gcloud config list projectList projectsgcloud config list, gcloud config list projectShow project infogcloud compute project-info describeSwitch projectgcloud config set project <project-id>Set the active accountgcloud config set account <ACCOUNT>Set default regiongcloud config set compute/region us-westSet default zonegcloud config set compute/zone us-west1-bList configurationsgcloud config configurations listActivate configurationgcloud config configurations activate Google Cloud... Continue Reading →
How to get Higher package
Must Have Notes:Never give expected ctc as a number at the beginning [Specially through phone call]Don't ask unrealistic numbersDon't show poor attitude while negotiating Don't talk about financial commitments [No company or recruiter interested in your financial conditions]Don't fall into any argument or fight [Politely refuse the offer and say thank you for this opportunity]Never... Continue Reading →
Top 7 GCP tools to learn for FREE.
Top 7 GCP #dataengineering tools to learn for FREE.....1. Google Bigqueryhttps://lnkd.in/g4Pvu8aq2. Google cloud Dataprochttps://lnkd.in/gZbJV_8shttps://lnkd.in/gkDeVqtb3. Google cloud Dataprephttps://lnkd.in/gF4G3uAK4. Google Cloud composerhttps://lnkd.in/gjfnYb3whttps://lnkd.in/grGTQYtT5. Google cloud Data Fusionhttps://lnkd.in/gfmxapqP6. Google Data Studiohttps://lnkd.in/gus75kYW7. Google cloud Dataflowhttps://lnkd.in/gyxKXaGU 8. Datawarehousing with Big query https://youtu.be/ZVgt1-LfWW4?si=dPVaNH9LgU-Wfo7s complete GCP Full course for FREE....https://lnkd.in/gi48NG3zResources are short and crispy, and definitely recommended.
Read CSV File by Spark
---------------Spark Interview Questions------------📕How to read a csv file in spark?Method 1: ---------------spark.read.csv("path")df=spark.read.csv("dbfs:/FileStore/small_zipcode.csv")df.show()---+-------+--------+-------------------+-----+----------+|_c0| _c1| _c2| _c3| _c4| _c5|+---+-------+--------+-------------------+-----+----------+| id|zipcode| type| city|state|population|| 1| 704|STANDARD| null| PR| 30100|| 2| 704| null|PASEO COSTA DEL SUR| PR| null|| 3| 709| null| BDA SAN LUIS| PR| 3700|| 4| 76166| UNIQUE| CINGULAR WIRELESS| TX| 84000|| 5| 76177|STANDARD| null| TX| null|+---+-------+--------+-------------------+-----+----------+Method 2 :--------------df=spark.read.format("csv").option("inferSchema",True).option("header",True).option("sep",",").load("dbfs:/FileStore/small_zipcode.csv")df.show()+---+-------+--------+-------------------+-----+----------+|... Continue Reading →
AWS Certification
FREE AWS Certificate by Amazon that you can't miss in 20231. Getting Started with Data Analytics on AWS🔗https://lnkd.in/dwRhRAzM2. Practical Data Science on the AWS Cloud Specialization🔗https://lnkd.in/d3-3GZbG3. Getting Started with AWS Machine Learning🔗https://lnkd.in/dhAp-Vjh4. Introduction to Machine Learning on AWS🔗https://lnkd.in/detfDCWA5. Hands-on Machine Learning with AWS and NVIDIA🔗https://lnkd.in/dgGvATq26. AWS Fundamentals Specialization🔗https://lnkd.in/dSV9jhRz7. Building Modern Python Applications on AWS🔗https://lnkd.in/dQAinFGy8. AWS... Continue Reading →
Free Spark Course
Don't pay for Apache Spark Course because it is in demand.You can learn for free here......1. Install spark from here....https://lnkd.in/gx_Dc8phhttps://lnkd.in/gg6-8xDz2. Learn spark Basics from here--https://lnkd.in/g-gCpUyihttps://lnkd.in/gkNhMnTZhttps://lnkd.in/gkbVB6YX2.1 Learn spark with Scala from here:https://lnkd.in/gtrZAmn42.2 Learn spark with python from here:https://lnkd.in/gQaeSjbH3. Learn pyspark from here:https://lnkd.in/g6kyihyW4. Work on Spark projects from here..https://lnkd.in/gE8hsyZxhttps://lnkd.in/gwWytS-Qhttps://lnkd.in/gR7DR6_5https://lnkd.in/gzngHhrChttps://lnkd.in/gACn6bK85. Finally list down your projects Here.....https://github.com/I highly recommend... Continue Reading →
Big Data Resources
#Resources Referred by me for Big data Technologies These resources are available for free in YouTube, which helped me to crack CISCO.. and for you to crack product based companies also..1.Hadoop ,sqoop and Hive concepts by Saif shaik:https://lnkd.in/ewyYweTJ2.pyspark concepts in depth by karunakar goud:https://lnkd.in/eNtFkxmd3.Another spark playlist which useful Raja's Data Engineering channel.https://lnkd.in/eqiy7dBS4. Hadoop and Kafka... Continue Reading →
