Match score not available

Data Engineer

Remote:

Full Remote

Contract:

Freelance

Salary:

30 - 50K yearly

Experience:

Mid-level (2-5 years)

Work from:

India

Offer summary

Qualifications:

1 to 3 years of experience, Experience in data pipeline development, Proficiency in Apache Spark and SQL, Hands-on with Airflow and Kubernetes, Familiarity with Progress and Vector databases.

Key responsabilities:

Design and maintain scalable data pipelines
Collaborate with data scientists and engineers
Monitor and optimize data processing systems
Implement data security and governance best practices
Automate processes for business intelligence

Techgencia Private Limited Startup https://www.techgencia.com/

11 - 50 Employees

See more Techgencia Private Limited offers

Job description

Your missions

Company Overview

TechGencia Private Limited embodies excellence, passion, and flexibility. As a premier software development company based in Kerala, India, we deliver world-class IT consulting, offshore software development, testing, and user interface design services. Our diverse projects range from simple information systems and websites to complex enterprise architectures, both desktop and web-enabled. With a commitment to on-time, within-scope, and cost-effective service delivery, TechGencia upholds a quality-focused approach, making us one of the best software development companies in Kerala. Visit us at www.techgencia.com

Job Overview

We are seeking a Junior Data Engineer with a minimum of 1 to a maximum of 3 years of work experience to join our dynamic team on a freelance, remote basis. The role is open to candidates located in Thiruvananthapuram, Kollam, Cochin, Kozhikode, Malappuram, and Palakkad districts. The ideal candidate will have experience in data pipeline development and toolsets, specifically using technologies such as Apache Spark, Airflow, Kubernetes, SQL, Progress, and databases like Vector and relational databases.

Qualifications and Skills

Experience in developing and maintaining data pipelines, ensuring data integrity and performance.
Proficiency in using Apache Spark for data processing and analytics tasks.
Hands-on experience with Apache Airflow for orchestrating complex data workflows.
Strong knowledge of Kubernetes for container orchestration and efficient large-scale data processing.
Proficiency in SQL for querying relational databases and managing data sets.
Experience with Progress and Vector databases for high-performance data management.
Ability to troubleshoot data-related issues and perform root cause analysis.
Excellent problem-solving skills and a collaborative attitude towards team-oriented tasks.

Roles and Responsibilities

Design, develop, and maintain scalable data pipelines for optimal data integration and quality.
Develop and maintain platform development using Python and related technologies.
Collaborate with data scientists and other engineering teams to integrate new data sources.
Ensure the reliability and performance of data processing systems, monitoring and optimizing workflows.
Implement best practices for data security, governance, and compliance.
Automate manual processes and optimize data delivery to facilitate business intelligence initiatives.
Stay updated with emerging technologies and industry best practices in data engineering.
Participate in code reviews, ensuring code quality and sharing knowledge with peers.
Contribute to the continuous improvement of data engineering processes and infrastructure.

Implement and manage data processing, data pipelines, and analytical frameworks

Develop and integrate APIs for data interchange between different systems or services.

Train, retrain, and monitor machine learning systems and models as needed

Construct optimized data pipelines to feed machine learning models

Extend existing machine learning libraries and frameworks