Profasee - Sr Data Engineer

Profasee https://profasee.com/ - Team: Data Engineering (reporting to CTO)

What we are looking for.

We are looking for an entrepreneurial and business minded Python Engineer to help us disrupt the e-commerce industry. We believe it is time for technology and data to help merchants deliver the best pricing to their customers, no matter what. If you are looking for an opportunity to join a start-up led by a proven founder in an industry that is growing exponentially and an opportunity to work alongside a small and focused Data Science (DS) & Machine Learning (ML) Engineering team, then this is the opportunity to work on something not only exciting and fun, but also creative in ways that most people will never experience in their lifetime.

What you will do.

You will work side by side with e-commerce experts and the DS / ML team to build, test, and maintain systems that collect, manage, and convert raw data into usable information for the DS / ML team to interpret. You will be responsible for building and maintaining data pipelines, storage systems, and APIs to access the data. You will also provide support to the ML engineering team to help with data engineering.

The ideal candidate writes well-tested code that is easy to read and understand, communicates effectively with team members, asks for help and feedback when needed, and shares knowledge.

• Write code and tests for pulling data from 3rd party integrations and loading into a structured database for each data source (ETL).

• Provide API access to the data collected from 3rd parties.

• Analyze data sources to help decide which are useful for the DS / ML team.

• Maintain the data pipeline architecture, automation, scalability, and error handling.

• Monitor pipeline performance and stability.

Who you are.

• You enjoy building the tools and systems for delivering a production product.

• You are comfortable in a small, dynamic team, where you will be designing and leading initiatives.

• You are self-motivated and comfortable working in a 100% remote environment.

• You can balance the need to move quickly with the desire to build for the future.

• You are willing to bring your expertise to the table and willing let others bring their expertise as well.

Requirements

• 5+ years of software development experience

• Proficient with Python programming language

• Proficient with SQL programming language and databases (PostgreSQL, MySQL, etc)

• Proficient integrating with APIs, data formats (JSON, CSV, XML, etc), rate limiting, and error handling.

• Experience with distributed task queues (i.e. Celery), multiprocessing, job scheduling.

• Understanding of the infrastructure used to build a production application (ex. NGINX, RabbitMQ, Error and Performance monitoring tools, AWS services such as ECS/EKS, etc.)

• Strong communication skills, especially with non-software developers

• Experience with containerization (Docker/Kubernetes)

• Experience designing and delivering the architecture required to run code in production

• Experience with ETL pipelines

Good to have

• Experience working in early-stage startups and/or developing your own projects

• Experience with NoSQL and non-relational databases

• Experience with machine learning and data science.