Unlock Premium Benefits Today!
🚀 Elevate your career with priority access.
Stand out to top recruiters with Jobgether Priority Access:
  • ✅ Top Match: We help you identify job offers where you're a perfect fit.
  • ✅ Exclusive Referral Service: Connect directly with recruiters as a Jobgether Certified Premium User.
  • ✅ Personalized Feedback: Receive feedback and expert guidance on applications, and have your profile manually reviewed.
Go Premium

spark developer

Remote: 
Full Remote
Contract: 
Full time
Work from: 
India

Offer summary

Qualifications:

Bachelor's or Master's degree in Computer Science or related field., 3-5 years of experience in Apache Spark development or data engineering frameworks., Proficiency in Java or Scala, with experience in Python being a plus., Familiarity with big data technologies and tools such as Kafka, NoSQL databases, and ETL processes..

Key responsibilities:

  • Develop scalable distributed architecture as defined by the tech lead.
  • Analyze and assemble large data sets to meet functional and non-functional requirements.
  • Create and optimize ETL scripts for big data sources and automate reporting processes.
  • Collaborate with stakeholders to ensure all end-to-end requirements are met and adhere to the Software Development Life Cycle (SDLC).

CodersBrain logo
CodersBrain SME https://www.codersbrain.com/
201 - 500 Employees
See all jobs

Job description

JD for Spark developer
Experience:3 to 5 Years
Work Location: Bangalore 
  
Responsibilities 
 .   As a Spark developer you will manage the development of scalable distributed Architecture
defined by the Architect or tech Lead in our team. 
 .   Analyse, assemble large data sets to designed for the functional and non-functional
requirements. 
 .   You will develop ETL scripts for big data sources. 
 .   Identify, design & optimise data processing & automate for reports and dashboards. 
 .   You will be responsible for workflow optimizations, data optimizations and ETL optimization
as per the requirements elucidated by the team. 
 .   Work with stakeholders such as Product managers, Technical Leads & Service Layer
engineers to ensure end-to-end requirements are addressed. 
 .   Strong team player to adhere to Software Development Life cycle (SDLC) and
documentations needed to represent every stage of SDLC. 
 General Qualifications: 
 .   BE/BTech/MCA/MTech/MSC with 3-5 years of experience with Apache Spark development
or in alternative Data Engineering & analytics frameworks. 
 .   Overall experience in the field Java/Scala as a Backend developer could be between 3 years. 
 .   Working experience in Bigdata is important. 
 Technical Skills 
 .   Programming experience in Java/Scala is needed. Experience in Python also will have an
advantage. 
 .   Writing ETL Stack in scalable & optimized fashion using Apache Spark/Hadoop/Kafka etc. 
 .   Should have working experience in writing distributed & optimized Apache Spark jobs for
various Machine Learning Algorithms. 
 .   Experience building and optimizing data pipeline and data sets is essential. 
 .   Should have worked with Kafka, at least one of No SQL databases (e.g. Cassandra,
MongoDB, elastic DB) and at least one of RDBMS (e.g. MySQL) 
 .   Working experience to use cache such as Redis, Apache Ignite, Hazelcast will advantage. 
 .   Working knowledge of Docker and Kubernetes will be a plus. 
Kindly find the below inputs from customer to get quality profiles
 Include the criteria apart from Spark à Experience in No sql databases (elastic search OR
Cassandra OR MongoDB), Apache Kafka.
 Look for Spark Streaming (people who have used Sqoop, likely doesn’t know Spark Streaming).
 Scala or Java based work experience using Spark is a MUST. Python usage with Spark is
optional, not must.

Spark streaming ,Kafka experience is mandatory for this requirement. Kindly add more profiles with
Spark Streaming, Kafka, Nosql experience

Required profile

Experience

Spoken language(s):
English
Check out the description to know which languages are mandatory.

Hard Skills

Apache SparkApache KafkaJava (Programming Language)Scala (Programming Language)KubernetesRelational Database Management SystemsNoSQLDocker (Software)Big DataETL (Extract Transform Load)Data ArchitectureApache IgnitePython (Programming Language)HazelcastApache SparkRedis

Other Skills

  • Teamwork
  • Communication
Are you interested?

Related jobs

See more jobs