Match score not available

ML Data Engineer

Remote:

Full Remote

Work from:

Peru

Offer summary

Qualifications:

Bachelor's degree in computer science or related field., 7+ years in data engineering, 4+ focused on ML., Experience with Databricks and Apache Spark., Proficient in Python and SQL..

Key responsabilities:

Develop and maintain feature engineering pipelines.
Design ETL/ELT pipelines and integrate diverse data sources.

Encora Inc. XLarge http://www.encora.com

5001 - 10000 Employees

See all jobs

Job description

Important Information

Location: Peru, Colombia, Costa Rica, Bolivia

Work Mode: Remote

Job Summary

As a ML Data Engineer (12836), you will be responsible for designing, developing, and maintaining high-quality software solutions. You will collaborate with cross-functional teams to understand business requirements and translate them into scalable and efficient software applications. Your role will involve leading technical projects, mentoring junior engineers, and continuously improving software development practices to ensure the delivery of robust and reliable software systems.

Responsibilities and Duties

Feature Engineering & Data Integration: Develop and maintain feature engineering pipelines using Databricks to support ML models effectively.
Data Pipeline Development: Integrate diverse data sources (e.g., clickstreams, user behaviour, demographic data) to create user behaviour features/profiles for complex ML tasks
Medallion Architecture: Design and implement ETL/ELT pipelines aligned with the bronze, silver, and gold layers of the medallion architecture.
Model Support: Build data pipelines to support ML model training, calibration, and deployment, leveraging MLflow for experiment tracking and performance monitoring.
Query Optimization & Low-Latency Pipelines: Design low-latency, production-ready data pipelines to support real-time and batch model inference.
CI/CD Practices: Apply CI/CD principles for seamless pipeline deployment
Data Governance: Ensure pipelines comply with security and regulatory standards, particularly for handling PII, and maintain metadata and master data across the data catalogue.
Collaboration: Work closely with ml scientists, ml engineers, and other stakeholders to align data transformation with business objectives.

Qualifications and Skills

Bachelor's degree in computer science, software engineering, or a related field.
Extensive experience in software development with a focus on designing and building scalable applications.
Professional/ Advanced English skills.
7+ years in data engineering and at least 4+ years focusing on ML feature engineering, ETL pipeline development, and data preparation for ML.
Proven experience managing pipelines on Databricks using Apache Spark, with a strong understanding of the medallion architecture.
Familiarity with ML lifecycle management, with MLflow experience as a strong plus and advanced skills in Apache Spark (PySpark) for big data processing and analytics.
Proficient in Python for data manipulation and SQL for query optimization
experience building pipelines for real-time and batch model serving in production environments, and knowledge of CI/CD practices for ETL/ELT pipeline development.
Expertise in metadata and master data management within technical data catalogues.
Understanding of data security and compliance, especially with sensitive data like PII

About Encora

Encora is a global company that offers Software and Digital Engineering solutions. Our practices include Cloud Services, Product Engineering & Application Modernization, Data & Analytics, Digital Experience & Design Services, DevSecOps, Cybersecurity, Quality Engineering, AI & LLM Engineering, among others.

At Encora, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.