AI Engineer, Evaluation

About the Team

The AI Platform team is a newly formed team at the center of Mixpanel's AI-first analytics vision. With a greenfield charter, we're building the infrastructure that accelerates and transforms AI product development at Mixpanel, both internally and for our customers.

We provide the tools to improve AI products, enabling internal teams and customer agents to build things that were previously unimaginable. We build shared infrastructure that is essential for developing AI features with confidence and speed, and that gives every agent the tools, context, and quality guarantees to act autonomously on behalf of users.

Some examples of what we are building:

• Agent Optimization Framework: A system that optimizes AI products for quality, speed, and cost, using metrics from production and evals, given the context of the invocation.
• AI Agent Integrations and Accessibility: Products and tools that bring the power of Mixpanel to wherever it is most effective for our customers, including a Mixpanel slackbot, MCP, and a public skills library.
• AI Engine: Services that centralize core AI and LLM functionality at Mixpanel in order to accelerate AI development, compound the value of AI efforts, and provide customers with a consistent experience.

Role Overview

We are looking for an experienced and driven Senior Software Engineer to join our AI Platform team. You will be responsible for building the scalable, secure, and reliable infrastructure that accelerates AI agent development. You will be a leader and key contributor in a small, fast paced, newly formed team with a mandate to empower AI development across the company.

Responsibilities

• Platform Architecture: Design and develop the core backend services, APIs, and microservices that enable product teams to easily and securely leverage AI models.
• Agen