Six months ago, a hiring manager at a Fortune 500 financial services firm told us something that we've heard from almost every client since: 'We've interviewed 40 AI engineers and I still don't know if any of them can actually do the job.'
The AI engineer title has been inflated faster than any job title in recent memory. Three years of rapid AI adoption has created a massive gap between what candidates claim and what they can deliver. Here's how to close that gap in your hiring process.
The Three Types of 'AI Engineers'
When a candidate says they're an AI engineer, they usually mean one of three things:
Type 1: API Integrators — They've built applications on top of OpenAI, Anthropic, or other LLM APIs. They understand prompt engineering, context management, and tool use. They can ship fast. They cannot train models, fine-tune on custom data, or build infrastructure at scale.
Type 2: ML Engineers — They understand the math, have trained models, can work with PyTorch or TensorFlow, and know how to evaluate model performance. They may or may not have production deployment experience.
Type 3: AI Systems Engineers — They can do both — and they also understand the systems infrastructure (Kubernetes, GPU clusters, inference optimization) required to run AI at scale. These people are extremely rare and command $250K+ compensation.
Most companies hiring 'AI engineers' actually need Type 1 or Type 2, but they're interviewing as if they need Type 3. The result: candidates who are actually excellent Type 1 or Type 2 engineers get filtered out because they can't explain backpropagation from scratch.
The Interview Framework That Works
We've helped over 50 enterprise clients hire AI engineers. The interview framework that consistently identifies the right candidates has three components:
1. Work sample over whiteboard — Give candidates a realistic problem: 'Here's a customer support ticket dataset, here's our API, build something useful in 2 hours.' What they build tells you more than any coding interview.
2. System design for the actual job — Ask them to design the specific system you're building, not a generic recommendation engine. Listen for how they handle uncertainty and tradeoffs, not just technical knowledge.
3. Failure story — 'Tell me about an AI project that didn't work the way you expected.' Candidates who've shipped real AI systems have real failure stories. Candidates who've only done tutorials don't.
The AI talent market is competitive, but it's not as impossible as it feels. You're not competing for a finite pool of unicorns — you're competing for well-defined skills that, with the right hiring process, are very findable.