When you interview someone for a job, what are you really trying to find out?
You can learn a lot from a resume. Experience. Skills. Education. Accomplishments.
Then there are the standard interview questions: What are your strengths? What are your weaknesses? Where do you see yourself in five years?
I’ve never been particularly convinced that those questions tell us much. And at least for those who work directly with me, the interview process is much more interactive. I’m interested in who they are and how they’re likely to respond to the dangers, opportunities, and practical realities of work in the environment they’re likely to do it in.
Most people know what they’re supposed to say.
The more interesting questions are the ones that help you understand how someone actually behaves when the work gets difficult.
Will they keep pushing when something matters? Will they learn when they get something wrong? Can they change their mind when the evidence changes? Can they figure something out when they don’t already know the answer? Can they take the lead when it’s their turn, and step back when someone else has the better idea?
And, perhaps most importantly, do they care?
What you’re really trying to learn
I’ve seen different models for thinking about this. Some focus on culture. Others look at whether someone is organized, competent, and motivated.
Those can all be useful.
But underneath them are some simpler questions:
- How long will you keep pushing on something worthwhile before you give up?
- How hard is it to get you to change your mind when you’re wrong?
- How much do you learn from failure?
- How quickly can you learn something new?
- How comfortable are you letting someone else take the lead?
- How much do you care about the work and the people around you?
Notice that none of these questions is really about a person’s resume.
They’re about behavior.
And you don’t necessarily need to ask them directly. In fact, asking them directly probably isn’t very useful. Most people know what the “right” answer sounds like.
Instead, you listen to the stories.
What happened? What did they do? What did they learn? What did they get wrong? What would they do differently? Who did they give credit to?
You’re looking for evidence of how someone responds when reality doesn’t cooperate.
Because eventually, every job gets hard.
Plans change. Things break. People disagree. Someone makes a mistake. A project takes longer than expected. The original idea turns out to be wrong.
That’s when you find out who you actually hired.
The same questions are becoming relevant to AI.
An interesting parallel is emerging as we start working with smarter, more agentic AIs.
We often evaluate AI by asking whether it can produce the right answer.
That’s important. But it may not be enough.
As AI moves from answering questions to doing work, we’re increasingly giving systems goals rather than instructions. An agent might research a market, analyze a portfolio, write a report, monitor a process, or work through a multi-step problem.
Picture an agent monitoring a portfolio when the market gaps 4% overnight — does it flag the anomaly and ask what changed, or keep executing the plan as if nothing happened?
At that point, the interesting question isn’t only:
“Can it do the task?”
It’s also:
“How does it behave when the task doesn’t go according to plan?”
That sounds a lot like an interview.
How does the agent respond when it can’t find the information it needs? What does it do when two sources disagree? Does it recognize when its original assumption was wrong? Does it ask for help when it reaches the edge of what it knows? Does it adapt when circumstances change? Does it learn from feedback?
And perhaps the AI equivalent of “How much do you care?” is a little different:
Does the system consistently behave in ways that reflect the objective we’re actually trying to accomplish?
That’s a much more interesting question than whether it can generate a good response in a demo.
From prompts to behavior
This is one reason I think the shift from prompts to agents matters.
A prompt is mostly about getting a particular response, but an agent is about getting useful behavior over time.
Those require different kinds of evaluation.
If I’m hiring someone, I don’t expect to give them a 500-page instruction manual for every possible situation they’ll encounter. I want someone who understands the objective, knows the boundaries, can make reasonable decisions, and knows when to ask for help.
The same principle applies to AI systems.
The goal isn’t necessarily to make the instructions longer and longer. It’s to create systems that can operate within clear objectives, constraints, feedback loops, and escalation paths.
And then observe what they actually do.
Maybe that’s the better interview question.
The best interview questions don’t have to be the most clever; they just need to be the ones that reveal something about how a person behaves when the answer isn’t obvious.
As AI systems become more capable, we’ll need to get better at asking them the same kinds of questions.
Not just:
Can it do the job?
But:
How does it behave when the job gets complicated?
That’s a much more useful way to think about intelligence — whether it belongs to a person, an AI agent, or a system that combines the two.
So the next time you’re evaluating either one, what are you really asking?
Leave a Reply