What do coding interviews miss about real-world Applied AI and Forward Deployed Engineering?

A lot of engineering interview preparation focuses on solving a well-defined problem with clearly specified inputs and outputs.

But customer-facing engineering rarely starts that way.

Imagine receiving a client request like:

“We have thousands of support tickets. Can you use AI to identify the important ones and route them to the right teams?”

Before writing code, I’d want to understand:

• What actually makes a ticket important?
• What happens when the system makes a mistake?
• How do we evaluate success when labels are incomplete?
• When should an automated decision be escalated to a human?
• How would we detect problems after deployment?

The interesting engineering work is often in discovering the requirements, designing evaluations and deciding where automation should stop.

I’m exploring how to turn this kind of ambiguity into a practical engineering exercise.

For engineers who have worked in applied AI, solutions engineering or forward-deployed roles:

1. Which skills are hardest to assess through conventional coding exercises?
2. What real deployment or customer-integration challenges would make a good practical exercise?
3. If you’ve prepared for these roles, what did existing resources fail to cover?

Please avoid sharing confidential interview questions or employer information.

I’m researching the problem and may eventually develop an educational product. Nothing is being sold or promoted here.