Product Engineer (AI Agent Evaluation)

About the role

This is a Product Engineer role on an AI agent evaluation product, sitting at the intersection of engineering and community. You will ship features that help agent builders build more reliably while creating in public to close the loop between community needs and product development.

Key responsibilities

• Build and ship product features across evals, replay, cohorts, and developer experience, exercising strong judgment about what is essential versus superficial.
• Create in public on X and GitHub, posting demos and engaging with the AI engineering community to surface feedback that informs the product.
• Wear whatever hat the week demands, whether feature work, a marketing motion, customer demos, or community engagement.
• Stay close to target users—AI agent builders, evaluation practitioners, and developers using agent frameworks and custom runtimes.
• Close the loop between community feedback and product decisions, ensuring seamless distribution.

Required profile

• 3+ years building or shipping production software, with experience building or evaluating agents in production.
• Strong Python proficiency for production code.
• Hands‑on experience with agent frameworks such as LangGraph or equivalent tools.
• Experience building or working with evaluation systems for LLM or agent outputs.
• Experience with trace data, observability, or debugging agent execution.
• Experience shipping developer tools, SDKs, or APIs with attention to documentation, ergonomics, and demos.
• Open‑source contributions with measurable adoption by other developers.
• Active public presence on X, GitHub, or similar platforms demonstrating ability to communicate and market technical work.
• Comfort operating in early‑stage, high‑uncertainty environments and wearing multiple roles concurrently.

Required skills

• Python
• LangGraph (or similar agent frameworks)
• LLM evaluation systems
• Trace data handling
• Observability tooling
• Debugging agent execution
• SDK development
• API design

What we offer

• Salary range $125,000–$190,000 USD annually.
• Fully remote role with primary location in Munich, Germany.