Job: AI Response Evaluator
Work setting: On-site in Kebayoran Baru, Jakarta Selatan
We are looking for detail-oriented and analytical contributors to support next-generation AI development through human evaluation and annotation. In this role, you will assess and score AI-generated responses based on a combination of objective and subjective criteria, including accuracy, reasoning quality, clarity, safety, relevance, tone, and overall user experience. Your feedback will directly contribute to improving the performance and alignment of advanced AI models.
Responsibilities
- Evaluate and score AI-generated outputs using provided guidelines
- Compare multiple model responses and identify the strongest answer
- Provide concise justification and qualitative feedback
- Detect factual inaccuracies, hallucinations, bias, or unsafe content
- Support continuous improvement of annotation standards and workflows
- Collaborate with QA and project teams to maintain evaluation consistency
Qualifications
- Strong written communication and critical thinking skills
- Excellent attention to detail and analytical judgment
- Familiarity with AI, LLMs, prompt evaluation, or data annotation is a plus
- Ability to follow evolving guidelines in a fast-paced environment in English
Ideal Backgrounds
Candidates with experience in:
- AI data annotation
- Content evaluation
- Linguistics or translation
- Research or academic writing
- QA or moderation
- Technical or creative writing
- This is an exciting opportunity to contribute to the evolution of human-in-the-loop AI systems and help shape the future of intelligent technologies.
