Offer summary

Qualifications:

Bachelor's degree in Computer Science, Engineering, or a related field., Experience with AWS services such as S3, Glue, EMR, and Lambda., Proficiency in data processing technologies like Spark and Airflow., Strong skills in data transformation and management from various sources..

Key responsabilities:

Build and scale reliable data pipelines for real-time and batch processing.

Develop automated data workflows for analytics, reporting, and AI/ML models.

Ensure data quality and integrity across diverse data sources.

Collaborate with data science and business teams to align technical solutions with use cases.

Job description

Addepto is a leading consulting and technology company specializing in AI and Big Data, helping clients deliver innovative data projects. We partner with top-tier global enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. Our exclusive focus on AI and Big Data has earned us recognition by Forbes as one of the top 10 AI companies.

As a Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge technologies. Here are some of the projects we are seeking talented individuals to join:

Design and development of a universal data platform for global aerospace companies. This Azure and Databricks powered initiative combines diverse enterprise and public data sources. The data platform is at the early stages of the development, covering design of architecture and processes as well as giving freedom for technology selection.
Development and maintenance of a large platform for processing automotive data. A significant amount of data is processed in both streaming and batch modes. The technology stack includes Spark, Cloudera, Airflow, Iceberg, Python, and AWS.
Design of the data transformation and following data ops pipelines for global car manufacturer. This project aims to build a data processing system for both real-time streaming and batch data. We’ll handle data for business uses like process monitoring, analysis, and reporting, while also exploring LLMs for chatbots and data analysis. Key tasks include data cleaning, normalization, and optimizing the data model for performance and accuracy.

🚀 Your main responsibilities:

Build and scale reliable data pipelines for both real-time and batch processing, using technologies like Airflow and AWS Step Functions.
Develop automated data workflows that power analytics, reporting, and AI/ML models using services such as S3, Glue, EMR, Lambda, Redshift, and SageMaker.
Work with diverse data sources, ensuring consistent quality, integrity, and readiness for analysis across platforms.
Ingest, transform, and manage data from structured and semi-structured sources (including relational databases, cloud APIs, XML, JSON, and Hadoop/Spark).
Partner closely with data science and business teams to align technical solutions with real-world use cases.
Contribute to MLOps and CI/CD pipelines, helping streamline the deployment and monitoring of data products and models.
Continuously evaluate and improve existing processes for speed, efficiency, and scalability in a cloud environment.
Turn business problems into data-driven solutions, ensuring high performance, security, and scalability.