About ITRex

THE PLACE

ITRex - AI pioneers who build systems that actually work in the real world, not just in demos. We're 250+ people spread across the US and Europe, creating solutions for companies like Procter & Gamble and Shutterstock. We keep it simple, build it right, and focus on what works.

THE PEOPLE

We're the kind of people who don't ignore messages in Slack, who jump in to help when you're stuck on a problem, and who offer solutions instead of blame when things go sideways. We believe in openness, accountability, and having each other's backs. No office politics, no hidden agendas - just people who care about doing good work together and supporting each other to get there.

THE ROLE

We are looking for a strong C++ Engineer with hands-on experience deploying and optimizing modern AI models for production. The ideal candidate combines deep systems programming expertise with practical experience working with LLMs and modern deep learning architectures. Rather than building or training models from scratch, this engineer focuses on integrating, evaluating, profiling, and optimizing AI inference pipelines for high-performance on-device execution.

Responsibilities

Work on deploying machine learning models to edge devices using the frameworks: llama.cpp, ggml
Collaborate closely with researchers to assist in coding, training and transitioning models from research to production environments
Integrate AI features into existing products, enriching them with the latest advancements in machine learning

Requirements

Core Software Engineering

4+ years of professional experience in Modern C++ (C++17/20)
Strong knowledge of memory management, multithreading, profiling and performance optimization
Experience debugging low-level issues (memory leaks, fragmentation, OOM, concurrency)
Experience working with Linux development environments

AI Inference / ML Systems

Experience integrating machine learning models into production applications
Experience deploying and optimizing AI inference pipelines
Hands-on experience with AI inference frameworks such as: llama.cpp (strong plus), ggml (strong plus), ONNX Runtime, TensorRT / TensorRT-LLM, OpenVINO, MLC LLM, ExecuTorch, TVM

Experience profiling inference performance and optimizing memory usage and latency

Deep Learning Knowledge

Strong understanding of modern AI model architectures, including:

Transformer architecture
Large Language Models (LLMs)
Diffusion Models
Tokenization
Attention mechanisms
KV Cache
Quantization techniques
Model conversion and deployment

Practical AI Experience

Experience working with one or more of the following: LLM deployment, Computer Vision models, OCR models, Multimodal models, Speech models, Image generation models
Experience evaluating new models and integrating them into existing products is highly desirable

Nice to Have

CUDA
Vulkan Compute
Metal
OpenCL
Typescript
Python
Experience contributing to open-source AI infrastructure projects

Benefits

Why people stay

First, the foundation:

Remote flexibility: Work where and how you work best - we trust you to deliver
Fair compensation: Competitive salary + benefits that matter (medical, learning)

Then, the growth:

Ownership opportunities: See a problem worth solving? Own it. We back smart risks over bureaucratic safety
AI enhancement: We leverage AI to make you faster and stronger - complementing your abilities, not replacing them
Learning investment: English classes, professional development
Career progression: Real paths up, not just sideways shuffling

Finally, the people:

Responsive teammates: No ignored Slacks, no "not my problem" attitudes
Supportive culture: When you're stuck, people help. When things break, we fix them together
Human connections: Regular meetups, tech talks, and actual relationships beyond work

Curious? We are too. Let's talk

Originally posted on Himalayas

AI Inference Engineer QVAC at ITRex Group