Model & Prompt Evaluation Design Jobs

1 job found

Today

Product Designer

Meta Seattle, WA

Seattle, WA

Today

$201k - $278k/yr Full time Ai Agent Design Design Designer

Apply Now

Job Description

Meta Business Agent is seeking a Product Designer to bring an end-to-end multi-modal AI agent vision to life. The role focuses on business- and customer-facing agent experiences across chat, voice, and visual surfaces, including prompt design, evaluation strategy, and multimodal quality systems.

Responsibilities

Write and revise prompts that shape Meta agentic tools, features, and behaviors on a product surface.
Test the product surface, translate findings into prompt improvements, ship updates, and verify that the delivered experience matches the intended prompt behavior.
Package prompt limitations into training inputs, including graders for crisp behaviors and human-feedback questions with good/bad pairs for fuzzier tasks such as writing quality.
Partner with engineering and data science to define what strong agent output looks like across chat, voice, and visual surfaces, and drive the evaluation strategy to measure it.
Expand evaluation coverage beyond text to include voice and visual agent output, defining how quality is judged when the surface itself is conversational or visual.
Define quality bars that remain consistent across a long tail of businesses, supporting a single agent specification from small business through enterprise without bespoke rework.
Prototype new ideas and experiences quickly to help the team prioritize new workstreams based on impact and feasibility.
Define, evolve, and unify visual and interaction systems that scale across products and platforms.
Define evaluation strategy and build eval components such as golden sets and judges, both independently and in collaboration with data science and engineering.
Leverage familiarity with code to update language and prompts using internal tools, review changes to confirm the shipped experience, and build logic that adapts agent behavior to context.
Work with product and cross-functional partners to establish a strategy for multi-modal agent communication.
Present design strategy and rationale to leadership and cross-functional executives, shaping alignment on product direction and influencing org-wide priorities across product, engineering, research, and content strategy.

Requirements

Bachelor’s Degree in HCI, design, or other related fields, or equivalent work experience designing and building agentic experiences.
11+ years designing software products with a focus on user experience, interaction, and visual design aligned to business objectives.
Experience building and maintaining evaluation pipelines for LLM products, including graders, rubrics, comparison sets, regression suites, and the infrastructure that runs them across models.
Experience designing and building agentic experiences.
B2C or B2B experience.
Deep expertise in prototyping tools, with the ability to evaluate and advance design craft across different fidelity levels.

Preferred Qualifications

Experience adhering to and implementing responsible, ethical AI practices such as risk assessment, bias mitigation, and quality and accuracy reviews.
Experience integrating AI tools to optimize or redesign workflows and drive measurable impact, such as efficiency gains or quality improvements.
Experience turning product rubrics into training signal, including graders, human-feedback questions, or preference pairs.
Experience shaping agent behavior on voice or other multi-modal surfaces.
Demonstrated ongoing AI skill development, including prompt and context engineering or agent orchestration, and staying current with emerging AI technologies.
AI-native design fluency, including experience with model design, generative AI logic, and defining assistant behavior and tone at the systems layer, not only surface-level copy.
Front-end or notebook-to-app experience, and opinions about what makes an eval result legible at a glance.
Experience with both B2C and B2B is preferred.

Education

Bachelor’s Degree in HCI, design, or other related fields.

Compensation and Location

Location: Seattle, WA (onsite)
Salary: USD 201,000 - 278,000 per year

View full job page →

Get Model & Prompt Evaluation Design Jobs alerts

New Model & Prompt Evaluation Design Jobs, daily.