Research Engineer COMPL-AI

The Secure, Reliable, and Intelligent Systems (SRI) Lab is a research group in the Department of Computer Science at ETH Zurich. Our current research focus is on the areas of reliable, secure, robust and fair machine learning, large language models, probabilistic and quantum programming, and machine learning for code. Our work led to six ETH spin-offs, among others LatticeFlow AI.

LatticeFlow AI enables organizations to build performant, trustworthy AI applications. As a pioneer in AI validation, LatticeFlow has developed, COMPL-AI—the industry’s first comprehensive EU AI Act mapping in partnership with ETH Zurich and INSAIT, helping AI teams align with regulatory standards.

We are looking for Research Engineers to research and integrate SOTA benchmarks into the codebase, add documentation, and improve the overall usability of the benchmark suite with the greater goal of encouraging adoption and integration within a wider GenAI compliance and evaluation ecosystem.

This Research Engineering role will start with a 4-month contract, with the possibility to extend an additional year. Come and build the next generation of trustworthy AI.

AI safety and compliance are fundamental to ensuring that artificial intelligence (AI) systems are safe and aligned with societal values. COMPL-AI is the world’s first open-source compliance-centered evaluation framework for Generative AI models and allows companies and users world-wide to understand how their models stack up against the EU AI Act. The open-source framework includes a benchmark suite with 27 state-of-the-art benchmarks and an evaluation engine.

As a Research Engineer, your main responsibilities will include developing, testing, maintaining, and contributing to the GitHub COMPL-AI benchmark suite codebase. You will also be tasked with integrating relevant public SOTA benchmarks as they become available to increase coverage of the EU AI Act requirements. Additionally, you will collaborate with researchers at ETH Zurich and LatticeFlow AI to scope new features and capabilities. Furthermore, you will work alongside the wider evaluation ecosystem to enhance interoperability and integration with existing tools. Staying up-to-date with the latest LLM evaluation research, industry trends, and LLMOps will also be a key aspect of your role.

• Preferably holding a BSc/MSc in Computer Science, Engineering, Mathematics or related technical/scientific field. A PhD would be beneficial but not required.
• Experience and proficiency in Python, Pytorch, and the ability to write clean and maintainable code.
• Solid theoretical and practical background in machine learning and computer science.
• Strong communication and collaboration skills.
• Previous experience with LLMOps, LLM evaluation, and LLM testing frameworks  (e.g. openAI’s evals, Stanford’s HELM) is a plus.
• Active involvement in contributing or running open-source projects is a plus.
• Eligibility to work in Switzerland is required.

If this role and our mission resonate with you, but you're hesitant because you don't check all the boxes, don't let that hold you back. Apply for the role below, we’d love to meet you and discuss and share more!

• Be part of a keystone project contributing towards AI Safety and Compliance
• Freedom to engage with wider open-source community and the university ecosystem.
• Pursue an intellectually challenging job with a real impact.
• Work with domain experts and leaders in their field. LatticeFlow is a spin-off from ETH Zurich, one of the world's top 10 universities.