AI Evaluation Engineer/Researcher

Summary This position at Hum. Angle focuses on testing, comprehending, and refining artificial intelligence systems tailored for journalism, human security, humanitarian analysis, and African contexts. The role is pivotal in supporting the organization's strategic goal of developing locally grounded humanitarian AI that accurately interprets African conflict environments while ensuring human judgment remains central to decision-making processes.

Responsibilities

Acquire proficiency in contemporary methodologies for assessing large language models and diverse AI architectures. Plan and execute structured evaluation protocols under the guidance of senior researchers. Create and curate comprehensive evaluation datasets, test scenarios, scoring rubrics, and performance benchmarks.

Evaluate model responses to determine accuracy, contextual relevance, potential bias, safety compliance, and areas of failure. Assess how AI technologies interpret specific dynamics within Nigerian, Sahelian, and broader African conflict and humanitarian landscapes. Perform comprehensive literature reviews and replicate relevant scientific experiments.

Leverage Python and specialized research tools to automate assessment workflows and analyze outcome data. Document research methodologies, key findings, identified limitations, and ensure reproducibility of results. Collaborate with journalists, subject-matter experts, researchers, and engineers to integrate local insights into the evaluation framework.

Formulate independent research questions and design experiments with increasing autonomy.