Senior Software Engineer – Backend & Data Pipelines

Job Title: Principal Data Engineer

About Assembly Industries:

Talent is distributed but Opportunities are not - Assembly Industries is breaking that pattern by building an AI enabled talent platform that connects top-tier, highly skilled global professionals with innovative companies across the US. As a fast-growing startup, we are laser-focused on impactful growth, agile strategies, and exceptional results.

Role Overview:

We are seeking a Senior Software Engineer to build the core back-end of the Assembly Orchestrator platform, our system for creating and executing Standard Operating Processes (SOPs) at scale. You will design and implement Java-based micro-service APIs, power the data pipelines that sit behind them, and ensure that everything operates with high performance, reliability, and security in the cloud.

Key Responsibilities:

Backend API Development:

• Design and implement Java micro-services that expose REST/JSON (and GraphQL, where appropriate) endpoints for SOP definition, execution, and monitoring
• Define domain models, validation logic, and backward-compatible contracts
• Ensure concurrency safety, idempotency, and exactly-once semantics during high-burst traffic

Scalability & Deployment:

• Containerize services with Docker and deploy to AWS EKS/Kubernetes using Helm or Kustomize
• Establish CI/CD pipelines with blue-green or canary release strategies for zero-downtime rollouts
• Tune JVM parameters, thread pools, and HPA policies to meet sub-250 ms p99 latency SLOs

Data Pipeline Integration

• Publish domain events to Apache Kafka via transactional or outbox patterns
• Build or extend streaming jobs (Kafka Streams / Spark) that enrich and persist SOP data for analytics
• Optimize storage layouts (PostgreSQL, Parquet/S3, Elasticsearch) for fast, multi-tenant queries

Performance, Reliability & Observability

• Implement resilience patterns (bulkheads, circuit breakers, rate limiting) using Resilience4j or similar libraries
• Instrument systems with Prometheus metrics, Jaeger traces, and structured logs (ELK)
• Define SLOs/SLAs, create alert rules, and participate in on-call rotations and post-mortems

Security & Compliance

• Enforce least-privilege IAM roles, secret management (AWS KMS / Vault), and fine-grained RBAC in Kubernetes
• Automate evidence collection for SOC 2 and ISO 27001 audits

Collaboration & Mentorship:

• Participate in design and code reviews, share best practices, and support the growth of junior engineers

Technical Requirements:

• Languages & Frameworks: Java, Python 3.9+ (FastAPI/Django for auxiliary tooling), TypeScript/JavaScript
• Data & Workflow Tools: Apache Kafka, Kafka Streams / Spark, Temporal, PostgreSQL, Elasticsearch
• Infrastructure: AWS (EKS, S3, RDS, MSK), Kubernetes, Docker, Terraform
• Monitoring & Observability: Prometheus, Grafana, Jaeger, ELK Stack
• Security & Compliance: SOC 2, ISO 27001, Secret Management, Key Rotation

Minimum Qualifications:

• 7+ years in back-end or data-engineering roles, with at least 3 years building Java micro-services at scale
• Production experience with event-driven architectures and streaming pipelines (Kafka or equivalent)
• Hands-on deployment to AWS + Kubernetes and automation of CI/CD workflows
• Demonstrated expertise in performance tuning, fault tolerance, and observability for high-throughput systems
• Strong communication skills and a track record of collaboration in cross-functional teams

Preferred Qualifications:

• Experience with workflow engines such as Temporal or Airflow
• Background in real-time analytics and stream-join patterns
• Familiarity with machine-learning data pipelines (feature stores, online inference)
• Contributions to open-source projects
• Previous startup experience operating in fast-moving, ambiguous environments