Offer summary

Qualifications:

Strong experience with Hive, Spark, Scala, and Python., Proficient in data modeling concepts and SQL scripting., Hands-on experience with Spark Streaming or live stream processing technologies., Familiarity with cloud development (AWS, Azure, GCP) is a plus..

Key responsibilities:

Design and implement big data infrastructure and processing pipelines.

Troubleshoot and resolve technical issues related to data pipelines.

Oversee assigned programs and guide team members in their tasks.

Build Flask and Rest APIs for data processing and analysis.

Job description

Job Title: Senior Data Engineer

Job Description: Senior Data Engineer (Hive, Spark, Scala, and Python).

We are seeking a highly skilled data engineer to join our team. In this role, you will be responsible for designing and implementing scalable data pipelines to process large amounts of data. This will include extracting data from a variety of sources, transforming and cleaning the data, and loading it into Hive or Spark for analysis. You will also be responsible for optimizing the performance of these pipelines and ensuring their reliability. As a Senior Data Engineer, you will play a critical role in the design, implementation, and maintenance of high-performance big data infrastructure, systems, and processing pipelines. You will work with a team of data engineers to scale our systems to handle billions of records of structured and unstructured data.

Key Responsibilities:

 Design and implement big data infrastructure, systems, and processing pipelines

 Troubleshoot and resolve technical issues related to data pipelines

 Oversee assigned programs and guide team members

 Explore new technologies and learn new techniques to solve business problems creatively

 Build Flask and Rest APIs for data processing and analysis

 Experience in Data Modelling concepts and SQL script

 Hands-on experience with Spark Streaming or other live stream processing technologies

 Any Cloud development (AWS, Azure, GCP) would be a plus and added advantage

Required Skills and Experience:

 Strong experience with Hive, Spark, Scala, Python, Linux, and Shell scripting

 Strong understanding of data modelling concepts and SQL script

 Experience in building and maintaining high-performance big data infrastructure and processing pipelines

 Experience with Spark Streaming or other live stream processing technologies

 Any Cloud development (AWS, Azure, GCP) would be a plus and added advantage

 Strong problem-solving and analytical skills

 Excellent communication and team management skills

This is an exciting opportunity for an experienced Senior Data Engineer to take a lead role in a dynamic organization. If you are passionate about data engineering and want to work with a team of talented individuals, we would love to hear from you.

Required profile