Principal Data Engineer

Thank you for considering the Principal Data Engineer position at Reveal Health Tech. We are an early-stage IT startup based in the US and India, focused on leveraging technology to deliver transformative healthcare solutions.

ABOUT THE ROLE

We are seeking a skilled Data Engineer proficient in AWS services and Pyspark to join our dynamic team. As a Principal Engineer, you will be responsible for architecting, designing, implementing, and maintaining scalable data pipelines to support our growing data processing needs. The ideal candidate is independent, detail-oriented, and capable of performing data validation and debugging to ensure high data quality and reliability.

Find out more about our mission and why we started Reveal Health Tech on our website.

Requirements

Responsibilities

• Designing and implementing scalable, robust, and maintainable data pipelines using AWS services such as Glue, EMR, S3, Redshift, DynamoDB, Amazon Athena, and OpenSearch.
• Implementing data quality checks and validation processes to ensure accuracy, completeness, and integrity of data.
• Designing effective data models and architectures to optimize data processing and facilitate downstream Data Science and Machine Learning workflows.
• Utilizing data quality validation tools to automate and streamline validation processes.
• Collaborating with cross-functional teams (e.g., Data Science, Software Engineering) to understand data requirements and implement efficient data processing solutions.
• Monitoring and optimizing the performance of data pipelines.
• Debugging issues and resolving data-related problems in a timely manner.
• Documenting data pipeline architectures, processes, and procedures.
• Stay up-to-date with new technologies, tools, and best practices in the data engineering field.

Key Skills and Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent work experience).
• Proficiency in data modeling and architecture, leveraging AWS services to architect scalable and efficient data solutions.
• Proven experience as a Data Engineer or in a similar role.
• Strong proficiency in AWS services relevant to data engineering (e.g., AWS Glue, Lambda, S3, Redshift, DynamoDB, Amazon Athena, OpenSearch).
• Experience designing and building data pipelines from diverse source systems.
• Solid understanding of data lake and warehousing concepts and best practices.
• Proficiency in programming languages used for data manipulation and transformation (e.g., Python, SQL).
• Experience with relational databases like PostgreSQL (PSQL).
• Understanding of database systems, including schema design, SQL querying, and performance optimization.
• Experience implementing data quality checks and validation processes.
• Experience using Cloudformation and CI/CD for deployment.
• Familiarity with data quality validation tools and frameworks (Monte Carlo).
• Ability to work independently and as part of a team in a fast-paced environment.
• Excellent problem-solving skills and attention to detail.

Preferred Skills:

• AWS certification (e.g., AWS Certified Solutions Architect, AWS Certified Data Analytics Specialty).
• Highly skilled in big data technologies like Pyspark.
• Familiarity with serverless computing (e.g., AWS Lambda) and containerization (e.g., Docker).
• Experience with LLMs and machine learning is highly preferred

How you will enrich us?

• Energetic and enthusiastic
• Autonomous and self-motivated
• Growth mindset
• Embraces challenges
• Building new things gets your blood pumping!
• Curiosity and deep interest in the world
• Challenges the status quo constructively

Benefits

What do you get in return?

• Be part of a growing/amazing team - A great opportunity to be part of 0-1 of a new age technology services & product engineering company in a risk-adjusted environment with high upside for initial members.
• Trust over control - We believe in strong business fundamentals and possess vision for scale from day 1. Truly, people are our greatest winners and we want to make sure your full potential is met in the job.
• Numerous on-the-job-and-beyond learning opportunities - We already have a curated list of courses you can dive right in!
• Industry best compensation and benefits

Next Steps

• Send us your updated CV - if you can mention how you have enriched your previous organisation in a cover letter, that would be great!
• If we find your profile suitable, we will have our Talent personnel to reach out to you to understand your profile/interests and how best we can align mutually.
• You would have two tasks - a computer-based reasoning test and a technical case study followed by a job-based interview in the next rounds.
• Finally, you would have a chat with our founder to understand more about us and see if this is the right next career move!