Machine Learning Software Engineer

Jan is a desktop application that turns computers into thinking machines, enabling open-source LLMs and remote API AIs on Windows, Mac, or Linux. Since its January 2024 release, it has been downloaded over 500,000 times. With a vision to redefine human-machine collaboration, Jan aims to revolutionise our interaction with technology. It ensures complete ownership, privacy, and full customisation for its users.

Jan is looking for a Machine Learning Software engineer can help run Jan on a wide variety of consumer-grade hardware to server-grade hardware, to support the 300,000 people who have downloaded Jan since 1st Jan 2024.

Responsibilities

Ensuring Jan’s inference and training to run stably on the following dimensions:

• CPU Hardware:
  • Intel consumer and Xeon (incl. AVX-2, AVX-512, AVX instructions).
  • AMD Ryzen.
  • Apple Silicon.
• GPU/ AI specific Hardware:
  • Nvidia (CUDA, Triton Inference Server using either vLLM/ TensorRT-LLM Backends).
  • AMD (ROCm), AMD Ryzen AI.
  • Intel Arc (sycl, Big DL, Intel Extensions for Transformers), Intel NPU.
  • Apple Metal/ CoreML.
• Cross-platform model format (e.g. GGUF, Pytorch).
• Operating Systems (Windows, MacOS, Linux).
• Popular Inference engines (llama.cpp, vLLM, TensorRT-LLM).
• Popular Training framework (Pytorch, MLX).
• [Bonus] Edge training/ fine-tuning, micro training/ federated training on various hardware and software AI stacks.

Requirements

• Demonstrated software engineering skill and proven track in Python, C++.
• Ability to be communicative in English.
• Familiar with development tools and workflows such as Git, CI/CD, Agile.
• Experience with hardware optimization (CPU, GPU, NPU).
• Experience but not limited to LLMs, including popular foundation models like Llama2, Mistral, GPT2.
• [Bonus] Experience running LLMs/ AI models in production in server/ edge.

Benefits

• We pay an “all-in” pay and you will cover your own insurance/medical from the amount.
• 14 days leave (and unlimited sick days).
• Annual equipment budget (once 2 month probation has been completed).