Databricks_Pool Hiring

Job Description AWS Databricks

SN Required Information Details 1 Role Databricks Engineer 2 Required Technical Skill Set Databricks on AWS 3 No of Requirements 1 4 Desired Experience Range 8-10 years 5 Location of Requirement Desired Competencies (Technical/Behavioral Competency) Must-Have

• Designing and implementing highly performant data ingestion pipelines from multiple sources using Databricks on AWS
• Developing scalable and re-usable frameworks for ingesting large data sets and moving them from Bronze to silver and from Silver to Gold layer in databricks.
• Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
• Hands-on experience in query tuning, performance tuning, troubleshooting, and debugging Spark and/or other big data solutions
• Comfortable Spark programming in Python, Scala or Java
• Working with event based / streaming technologies to ingest and process data
• Experience with databases and data warehousing
• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprint
• Excellent communication skills

Good-to-Have

• Background in machine learning or working with Client tools and services

Sr. No Responsibility of / Expectations from the Role 1 Hands-on experience in architecting, developing, deploying, and operating large scale distributed systems using major components in the Hadoop ecosystem 2 Designing and implementing highly performant data ingestion pipelines from multiple sources using Databricks on AWS 3 Developing scalable and re-usable frameworks for ingesting large data sets and moving them from Bronze to silver and from Silver to Gold layer in databricks. 4 Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times 5 Working with event based / streaming technologies to ingest and process data 6 Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints.