Senior Quality Auditor, Artificial General Intelligence

The Senior Quality Auditor serves as a specialist who becomes the organization's deepest expert on 1-2 complex machine learning data workflows while maintaining hands-on audit execution. This role within the Machine Learning Data Operations job family leads Quality Audit Design (QuAD) teams for high-complexity workflows, handles the most ambiguous audit scenarios, and generates workflow-specific quality analysis that drives customer insights and workflow optimization.

Key job responsibilities

Workflow Expertise Development

• Become the organization's deepest expert on 1-2 assigned complex workflows through intensive pilot participation and continuous learning
• Participate in workflow pilots as an annotator to build foundational knowledge before production begins
• Develop comprehensive understanding of workflow conventions, edge cases, and quality requirements Team Leadership
• Lead QuAD teams of 3-5 Quality Auditors for Template 2 (High-Engagement Support) workflows
• Provide day-to-day leadership and make real-time quality decisions during iteration phases
• Handle escalations from team members and serve as the go-to expert for workflow-specific questions
• Mentor and develop Quality Auditor team members through coaching, feedback, and knowledge sharing Audit Execution
• Execute audits on the most ambiguous and complex scenarios, reducing escalations to Quality Specialists
• Maintain 60-70% hands-on audit execution to preserve expertise required for real-time decision-making
• Validate team audit decisions and ensure consistency across QuAD team members
• Support Template 1 workflows by auditing high-complexity scenarios requiring specialized judgment Analysis & Insights
• Generate workflow-specific deep dives and thematic analysis providing insights beyond conventions adherence scores
• Identify error patterns, convention evolution, and areas of ambiguity within assigned workflows
• Provide actionable recommendations for workflow optimization and quality improvement
• Partner with DSLs and customers on workflow-specific quality insights Strategic Collaboration
• Provide input to audit strategy design led by Quality Specialist I
• Collaborate with Quality Specialists on cross-workflow patterns and tactical improvements
• Participate in workflow design discussions with customers and stakeholders
• Support new-tool onboarding for operationalized and bootstrapped interfaces
• Track record of generating actionable insights from quality data.
• Bachelor's degree or equivalent - 2-4 years of experience in quality assurance, data operations, or a related field
• Experience working with machine learning data workflows or similar technical domains
• Hands-on experience in leadership and mentoring
• Strategic thinking about workflow optimization and process improvement
• Effective stakeholder communication skills with the ability to influence decision-makers and customers
• Domain expertise in STEM, language, or technical domains highly valued.
• Experience with explainable machine learning and artificial intelligence methodologies and tools
• Experience in leadership roles, overseeing teams conducting data analysis, process improvements, creative problem solving, and cross-functional collaboration
• Experience using data and metrics to drive actionable insights at scale
• Experience that includes strong analytical skills, attention to detail, and effective communication abilities
• Experience handling administrative details independently, while exercising good judgment in keeping team members adequately informed
• Familiarity with audit methodologies and quality frameworks
• Experience participating in pilot phases and workflow development Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.