Data Platform Engineer

Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data. Our business value and leadership has been recognized by various market research firms, including Forrester and Gartner. We are looking for top-notch talent as we continue to build the best analytics global consulting team in the world.

Requirements

High Level Design: Lead the solution design and implementation of end-to-end

data pipelines that support real-time streaming, data syndication, and integration

across multiple platforms, focusing on high availability, scalability, and

performance.

Technical Leadership: Manage and mentor a team of engineers, providing

guidance on technical approaches, troubleshooting, and problem resolution.

Lead the implementation of best practices for coding, testing, deployment, and

monitoring of data products.

Data Platform Modernization: Drive modernization efforts for existing data

platforms, ensuring smooth migration and integration with cloud-native

technologies and streaming data frameworks.

Streaming Technologies: Design and implement robust streaming data

pipelines using technologies such as Apache Kafka, Kinesis, Flume, and other

relevant tools to enable real-time data processing and integration.

Cloud Integration: Intimately understand AWS to architect and deploy cloud-

based data solutions that integrate seamlessly with the larger data ecosystem.

Performance Optimization: Identify bottlenecks in data workflows and

implement solutions to improve performance, reduce latency, and increase

throughput.

Reliability & Maintenance: Ensure that the data platform is reliable, resilient,

and able to handle high-volume data processing workloads, with a strong focus

on monitoring, incident management, and uptime.

Data Engineering Practices: Advise and enforce best practices around data

management processes, batch vs. real-time processing, data storage strategies,

and data quality.

Collaboration with Stakeholders: Work closely with cross-functional teams,

including data scientists, product managers, business analysts, and IT

operations, to deliver business value and ensure alignment of architecture with

organizational goals.

Technical Expertise

Data Engineering: Proven experience designing and implementing scalable,

high-performance data systems in cloud environments, specifically for large-

scale data ingestion, storage, and processing.

Streaming Technologies: Advanced knowledge of real-time data streaming

platforms such as Apache Kafka and AWS Kinesis.

Cloud Platforms: Expertise in working with AWS as well as a deep

understanding of the intricacies involved in moving cloud workloads to on-prem

having a common interface to be able to easily move workloads from cloud to on-

prem. Experience designing compute agnostic and multi-tenant solutions.

Databricks: Proficiency in Databricks for big data processing, particularly in

managing data pipelines and large-scale analytics workloads.

Programming: Strong hands-on experience with Scala and Java for developing

data-intensive applications, data pipelines, and stream processing systems.

Big Data Frameworks: Familiarity with Apache Spark for distributed data

processing.

Kubernetes: Experience with Kubernetes or other orchestration frameworks.

Experience writing infrastructure as code.

Benefits

Significant career development opportunities exist as the company grows. The position offers a unique opportunity to be part of a small, challenging, and entrepreneurial environment, with a high degree of individual responsibility.