Data Engineer
Offer summary
Qualifications:
Experience with dbt (Data Build Tool) or PySpark, with a preference for dbt., Proven ability to design and implement metadata-driven approaches for data ingestion., Strong knowledge of handling schema drift and implementing error handling within data pipelines., Familiarity with healthcare data (provider, payer, claims) is a plus..Key responsabilities:
- Work on ingesting a variety of healthcare data from multiple sources and vendors.
- Consolidate and streamline redundant data pipelines to improve scalability and performance.
- Collaborate with cross-functional teams to understand data requirements and identify areas of improvement.
- Leverage technologies like dbt and PySpark to enhance data processing capabilities.
Job description
Welcome to 10Pearls!
We believe in harnessing the power of technology for social good through our core values: Innovate, modernize and accelerate.
About 10Pearls
We are 10Pearls, an award-winning digital development company, that helps businesses with product design, development, and technology acceleration. We have a culture of innovation, uniquely designed to help companies transform, digitalize and scale by levering digital technology.
Wa are looking for a Senior Data Engineer to join our team.
Responsibilities
- Data Ingestion: Work on ingesting a variety of healthcare data (provider, payer, claims, etc.) from multiple sources and vendors, ensuring it is converted into Cohere’s common data model.
- Pipeline Optimization: Consolidate and streamline redundant data pipelines to improve scalability and performance.
- Metadata-Driven Design: Design and implement a metadata-driven approach for data ingestion, accounting for schema drift and providing robust error handling strategies.
- Collaboration: Collaborate with cross-functional teams to understand data requirements and identify areas of improvement in existing processes and pipelines.
- Scalability and Efficiency: Work towards making the data ingestion system more scalable and adaptable to changing data sources and formats.
- Technology Integration: Leverage technologies like dbt and PySpark to enhance data processing capabilities and improve overall data workflows.
Requirements:
- Experience with dbt (Data Build Tool) or PySpark. Experience with dbt is preferred, but PySpark experience will also be valuable.
- Proven ability to design and implement metadata-driven approaches for data ingestion.
- Strong knowledge of handling schema drift and implementing error handling within data pipelines.
- Familiarity with healthcare data (provider, payer, claims) is a plus.
- Ability to work with diverse data formats from multiple vendors and transform them into a unified data model.
- Experience with designing scalable and efficient data systems.
What we offer:
• Work from home
• Flexible Schedules
• Paid PTO
• Amazing People oriented organizational culture
• Challenging projects using the latest technologies with clients from the US and Canada
• Technology and Soft Skills Internal Training
• Online Courses from AdworldPrime
We thank you for applying to this job position, we’re more than thrilled to start reviewing your profile and great skills! This is the first step in our selection process, so you will be hearing back from our awesome recruitment team regarding the next steps 😀
10Pearls Team
Required profile
Experience
Other Skills
- Collaboration
Data Engineer Related jobs
Alovia
- Remote: Spain
- Full time
Swish Analytics
- Remote: United States
- Full time
Rapinno Health Care
- Remote: Texas (USA)
- Full time
Compass.uol
- Remote: Brazil
- Full time
Pagos Solutions
- Remote: Portugal
- Full time