17 open roles
AI Agent Evaluation Engineer
Job description
About LetitbexAI: LetitbexAI is a fast-growing AI-driven technology company focused on building intelligent, scalable, and enterprise-grade solutions. We work at the intersection of AI, data engineering, cloud, and business transformation, helping organizations unlock real value from artificial intelligence. Position: AI Agent Evaluation Engineer Experience: 8+ Years Notice Period: Can be considered up 15 Days Job Position (Title) : AI Agent Evaluation Engineer Experience: 8+ Years Job Summary :: We are seeking a highly motivated and technically proficient AI Agent Evaluation Engineer to join our growing AI team.
This crucial role will be responsible for defining, developing, and executing robust Agent evaluation frameworks and test strategies, with a significant focus on Responsible AI and Safety Evals, for our agents built using the Google Agent Development Kit (ADK). The ideal candidate will bridge the gap between AI development and reliable deployment, ensuring our agents are safe, ethical, effective, and meet high-quality performance standards.
The role will be of 70% Automation and 30% Manual Testing Required Skills & Qualifications: Experience: 8+ years in Software QA, with at least 2-3 years focused on testing or evaluating AI/ML systems, conversational agents, or Large Language Models (LLMs). Safety Evals Expertise (Mandatory): Direct experience in designing and executing safety evaluations (red teaming, adversarial testing), bias detection, and measuring toxicity/harmful content in generative AI models.
Agent/LLM Evals: Proven experience developing and running general evaluations (Evals) for LLM-powered applications knowing libraries like Py. Test (Must) Google ADK Familiarity (Mandatory): Direct or strong conceptual understanding of the Google Agent Development Kit (ADK) and its components. Programming: Strong proficiency in Python is mandatory for script development, data processing, and automation.
Cloud & MLOps: Familiarity with Google Cloud Platform (GCP) services relevant to AI/ML (e.g., Vertex AI) and integrating testing into MLOps workflows. Communication: Excellent analytical, problem-solving, and communication skills to articulate complex agent behaviors, risks, and safety findings clearly. Tools and Libraries: Langsmith, Deep.
Eval, Ragas, Giskard, Hugging face.
Description copied from LETITBEX AI TECH SOLUTIONS's careers page. Read the full posting before you apply.
More jobs at LETITBEX AI TECH SOLUTIONS
Market Research Analyst
LETITBEX AI TECH SOLUTIONS· Hyderabad, TG, IndiaSAP-ABAP Technical Consultant
LETITBEX AI TECH SOLUTIONS· Hyderabad, TG, IndiaSalesforce Developer
LETITBEX AI TECH SOLUTIONS· Bengaluru, Karnataka, IndiaPlaywright Automation Engineer
LETITBEX AI TECH SOLUTIONS· Hyderabad, TG, IndiaBusiness Analyst – Manufacturing AI Solutions
LETITBEX AI TECH SOLUTIONS· Hyderabad, TG, India
More jobs in Hyderabad
Security Engineer
SquareShift· Hyderabad, IndiaAssociate - Azure Fabric Developer
Oaktree Capital Management· HyderabadCollateral Management - Emerging Lead
State Street· Hyderabad, IndiaSoftware QA Tester-3
Centific· India, HyderabadEnablement Operations Effectiveness Specialist
Rimini Street· Hyderabad