Offer summary

Qualifications:

5+ years of experience in data engineering or related field, BS in Computer Science or related field required, Strong knowledge of SQL and Spark (using Scala), Excellent written and verbal communication skills..

Key responsibilities:

Build and modify Spark jobs to process high-volume streaming data

Develop and launch new features to meet evolving business needs

Collaborate with various teams to iterate on the data infrastructure roadmap

Maintain high standards of code quality and conduct code reviews.

Job description

Company Description

We’re looking for a skilled and proactive Data Engineer to join an innovative AdTech project focused on premium video advertising.

In this role, you’ll work with high-volume streaming data (10GB+ daily), building and optimizing pipelines using Spark, Scala, Python, AWS, and Databricks. You’ll help ensure fast, reliable, and scalable data infrastructure to support analytics, reporting, and public APIs.

If you love working with Big Data and want to influence the future of digital advertising — this role is for you.

CUSTOMER

Our USA client, a technology company building the next generation of advertising products and experiences for premium video. The mission is to provide the best advertising experience for consumers, the best monetization for premium publishers, and the best return for brand advertisers.

PROJECT

As a Data Engineer, you will be an important part of the team that owns our data infrastructure. This is no small responsibility, as an AdTech company, data is our lifeblood, and there's a lot of it. If you're excited by and have experience ingesting, processing, storing, analyzing, and working in a Big Data environment, then you are very likely the one to enjoy this role too!

Job Description

Build and modify Spark jobs (in Scala) to perform various tasks, from reading Kinesis streams using Spark Streaming, to joining and aggregating huge data sets, to integrating with third party data sources
Develop and launch new features to adapt to evolving business needs
Be an active and engaged owner of our data infrastructure
Be curious and seek to understand all aspects of our business
Maintain high standards of code quality, and encourage the same by providing constructive code reviews to collaborators
Troubleshoot and resolve issues, problems, and errors encountered across various
systems
Collaborate with Data Science, Product, Research, and Engineering teams to iterate on the roadmap
Gather requirements when underspecified

Qualifications

5+ years of experience in data engineering or related field
BS in Computer Science or related field required
Strong knowledge of SQL required
Strong knowledge with Spark (using Scala)
Working knowledge of serialization formats and their trade-offs (columnar vs row-based)
Experience debugging and optimizing Spark jobs
Familiarity with database fundamentals, such as ACID, snowflake schema, normalized/denormalized data
Must be a strong written and verbal communicator

WOULD BE A PLUS

Familiarity with columnar database, key-value stores, document stores, stream processing, time series databases, data warehouses, and OLAP
Experience working with HDFS, S3, GCP and BigQuery
Familiarity with Data Science tooling in Spark
Experience in the advertising industry is a plus
Experience with real-time analytics