71 open roles
AI Evaluation & Quality Engineer
Job description
AI Evaluation & Quality Engineer We. Quel is looking for an AI Evaluation & Quality Engineer for an assignment with one of our clients. You will make sure agentic workflows are measured honestly before and after they reach production – so the organisation knows what works, what does not, and why. About the assignment Workload: 100% Location: Hybrid Language: English Your responsibilities Design evaluation strategies and quality metrics for LLM agents (accuracy, task success, safety, cost, latency) Build and maintain evaluation datasets, test cases and automated regression tests Set up LLM-as-judge and human review processes where needed Run red-teaming and robustness tests, including prompt injection scenarios Monitor agent quality in production and feed findings back to development teams Report results clearly to product owners and stakeholders Requirements At least 3 years of experience in test automation, QA engineering or data science Hands-on experience evaluating LLM or ML solutions Strong Python skills Understanding of statistics and experiment design Fluent English, written and spoken Meritorious Experience with evaluation and observability tools (e.
g. Langfuse, Lang. Smith, Ragas, Deep. Eval, promptfoo) Experience with CI/CD pipelines for ML or LLM applications Knowledge of AI safety and the EU AI Act Background in industrial or manufacturing environments Swedish Who you are You are analytical, sceptical in a constructive way and thorough. You enjoy finding where things break, explain results in plain language, and help teams make better decisions based on evidence.
Applications are reviewed continuously – apply today!
Description copied from Wequel's careers page. Read the full posting before you apply.
More jobs at Wequel
Agentic AI Engineer
Wequel· Stockholm, SEAI Solution Architect – Agentic Systems
Wequel· Stockholm, SEAI Integration Engineer (MCP/API)
Wequel· Stockholm, SELLMOps / AI Platform Engineer
Wequel· Stockholm, SEInterim Head of QA Operations – start omgående
Wequel· Stockholm, SE
More jobs in Stockholm
Store Manager
Swarovski· Stockholm, SwedenExecutive Assistant & Office Manager
Intermediate Capital Group· StockholmClient Delivery Manager – Front Office Capital Markets Trading
Broadridge Financial Solutions· Stockholm, Kungsgatan 36Audio Video Planner
dentsu X· StockholmButikssäljare (25 timmar/vecka)
Levi Strauss & Co.· Ls Stockholm Nk, Stockholm, Sweden
