Test Analyst - AI Assurance

• Strong understanding of GenAI system architecture (RAG / Agents)

• Implement python-based AI Evaluation frameworks like Ragas, Deep. Eval, etc
• Perform Model & Agent quality evaluation using tools like Arize, Lang. Fuse, ConfidentAI, etc
• Perform Responsible AI validation & report RAI metrics
• Golden dataset curation & Synthetic dataset generation
• Ability to instrument & establish AI Observability / Tracing using OTEL libraries Expert in Python