Turn AI capability into product decisions that ship.
You will sit between what clients need and what AI can credibly deliver, turning fuzzy goals into clear scope, measurable outcomes, and the evaluation that proves a feature actually works. This is a modern, AI-native analyst seat: equal parts discovery, data, and judgment about where AI earns its place and where it does not.
What you'll do.
-
Shape product requirements for AI-enabled features — define the problem, the scope of a sprint, and what "good" measurably looks like.
-
Design and run evaluations for AI behavior: build the test sets, define the metrics, and report honestly on quality, not vibes.
-
Analyze product and usage data to find where AI compounds value — and where it adds risk or noise.
-
Work directly with engineers and clients to keep each 7-day sprint pointed at the highest-leverage outcome.
You should have.
-
Experience as a product analyst, product manager, or data analyst on software products.
-
Strong analytical skill — comfortable with SQL and reasoning from data to a recommendation.
-
A working, hands-on understanding of modern AI: what LLMs and agentic systems can and cannot reliably do.
-
Able to write crisp requirements and success criteria that an engineer can build against.
-
Clear communicator who works well async across time zones.
- Experience designing LLM evaluations or building eval harnesses.
- Familiarity with prompt design and agentic workflows.
- A technical enough background to read code and talk to engineers as peers.
- SQLClaudePythonAnalytics