Project details
Recommended projects
AI Agent Evaluation Analyst (m/f/d)
We are looking for an Freelance Agent Evaluation Analyst to take ownership of quality, structure, and insight across the project. This role goes far beyond task-checking - it’s about critical thinking, systems-level analysis, and ensuring clarity, reliability, and consistency at scale. You’ll work as both a hands-on evaluator and an analyst, collaborating with domain experts, delivery managers, and engineers. Beyond reviewing outputs, you’ll be expected to understand the “why” behind the work, identify logical gaps or inconsistencies, and propose meaningful improvements.
This is a flexible, impact-driven role where you’ll have space to grow, contribute ideas, and help shape how evaluation and quality are scaled across the project.
This role is especially well-suited for:
Analysts, researchers, or consultants with strong structuring and reasoning skills Junior product managers or strategists curious about AI and evaluation work Smart problem-solvers (students or early-career professionals) who enjoy digging into logic, systems, and edge cases
You do not need a coding background. What matters most is curiosity, intellectual rigor, and the ability to evaluate complex setups with precision.
What you’ll be doing
- Fully own the QA pipeline for agent evaluation tasks;
- Review and validate tasks and golden paths created by scenario writers and experts;
- Spot logical inconsistencies, vague requirements, hidden risks, and unrealistic assumptions;
- Provide structured feedback and ensure quality alignment across contributors; Train, onboard, and mentor new QA team members;
- Collaborate with domain experts, delivery managers, and engineers to improve test clarity and coverage;
- Maintain and improve QA checklists, SOPs, and review guidelines;
- Contribute to test planning, prioritization, and quality benchmarks;
- Take initiative to suggest new approaches, tools, and processes that help scale validation and analysis.
AI Agent Evaluation Analyst
For an AI lab we are looking for AI Agent Evaluation Analyst to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities
- Reviewing evaluation tasks and scenarios for logic, completeness, and realism.
- Identifying inconsistencies, missing assumptions, or unclear decision points.
- Helping define clear expected behaviors (gold standards) for AI agents.
- Annotating cause-effect relationships, reasoning paths, and plausible alternatives.
- Thinking through complex systems and policies as a human would to ensure agents are tested properly.
- Working closely with QA, writers, or developers to suggest refinements or edge case coverage.
AI Evaluation Consultant (m/w/d)
We are seeking an analytical and technically-minded professional to:
- Evaluate AI outputs and processes
- Ensure quality, accuracy, and reliability
- Identify logical errors, risks, and structural inconsistencies
- Provide actionable insights and recommendations to the team
Ideal candidates:
- Consultants, auditors, analysts, data researchers, or business/technical analysts with strong reasoning skills
- Professionals curious about AI, process improvement, and quality evaluation
- Problem-solvers who enjoy analyzing complex systems, logic, and scenarios
Key Responsibilities:
- Lead evaluation of AI outputs and related processes
- Review tasks against expected/ideal scenarios; identify gaps and risks
- Provide structured, actionable recommendations to engineers, domain experts, and managers
- Maintain and improve evaluation guidelines, checklists, SOPs
- Suggest new approaches, tools, and processes to enhance AI evaluation
Business Analyst – SAP S/4HANA Output Management (m/f/d)
- A company is looking for an experienced Business Analyst to support the transformation from SAP ECC to S/4HANA Utilities.
- The project goal is to analyze, document and optimize output and archiving processes, as well as to create functional designs and specifications.
- The analyst will work closely with product owners, IT and business units to align feasibility, effort and prioritization of requirements.
Freelance Automotive Engineer (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Chemistry Expert for AI Model Training (m/f/d)
An AI lab is looking for a freelance chemistry experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in chemistry contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Evaluate AI models for chemistry applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Freelance Electrical Engineer with Python Experience (m/w/d)
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Mechanical Engineer with Python Experience (m/w/d)
For an AI lab we are looking for Mechanical Engineer with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Mechanical Engineering, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Civil Engineer with Python Experience (m/f/d)
A company is looking for a freelance Civil engineering experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in civil engineering contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
Key responsibilities:
- Evaluate AI models for civil engineering applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Senior Project Manager Customer Interaction
A company is looking for support in the project to evaluate, implement and further develop quality surveys in digital channels. The goal of the project is to increase customer satisfaction in digital channels, evaluate, implement and enhance survey methods to enable consistent measurement of customer satisfaction across all channels. Improvement opportunities should be identified and implemented.
The role includes advising, developing and implementing measures to collect and improve customer satisfaction in digital channels. Main tasks:
- Advising on survey methods to gather customer experience and quality in digital channels, market standards, benchmarks and future direction.
- Developing a future model for quality in digital channels, relevant KPIs and survey methods as well as standard processes.
- Implementing the agreed measures, including interface management and coordination with technology partners and social partners.
- Testing the implemented measures to ensure they meet the required criteria.
- Consolidating and listing the existing and missing customer survey methods/quality KPIs across all responsible digital channels.
- Advising on the preparation of decision papers and implementing the necessary measures.
- Identifying improvement opportunities and developing a standard process for transparency and execution.
Freelance Physics Expert (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Statistics Expert with Python Experience (m/f/d)
For an AI lab we are looking for Statistics Expert with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
Expert for Setting Up a Call Center
A health insurance company is urgently looking for an expert to set up a call center. The goal of the project is to develop a comprehensive roadmap covering all aspects of the call center setup. The expert will work closely with internal stakeholders to analyze requirements and develop a structured approach.
- Create a roadmap for setting up a call center
- Identify and select suitable systems, tools, and technologies
- Develop organizational and staffing structures
- Define necessary processes
- Plan realistic implementation phases
Chemist with Python Experience (m/f/d)
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as an AI Tutor in Chemistry, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
Physicist with Python Experience (m/w/d)
For an AI lab we are looking for phycists with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a phycist, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Design original computational physic problems that simulate real research workflows.
- Create problems requiring Python programming to solve (using numpy, scipy, sympy).
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks).
- Develop problems requiring non-trivial reasoning chains.
- Base problems on real research challenges or practical applications from physical practice.
- Verify solutions using Python with standard libraries.
- Document problem statements clearly and provide verified correct answers.
Project Manager Magazines / Magazine Production (m/f/d)
- Responsibility for coordinating and managing the entire production process of magazine publications
- Planning and monitoring issue structure, deadlines, ads, and workflows
- Close collaboration with editorial, publishing management, marketing, technology, sales, printers, and service providers
- Quality assurance for layouts, texts, and print approvals
- Cost calculation and organization of supplementary products (e.g. inserts, posters, add-ons)
- Active role in strategic projects, conferences, and launching new formats
Developer for Consent Management Implementation (m/f/d)
To replace the consent layers on our international brands’ websites that were previously handled by third-party CMPs, we will build new in-house layers so we can operate and deploy them ourselves. Strong skills in Typescript, Vue.js and standard web presentation techniques (HTML and CSS) are required. The goal is to deliver executable code that implements all requirements and includes automated tests to prove correct functionality.
What the assignment covers: The main focus is to develop elements for decision-making on the approach and to carry out measures throughout the project. This specifically includes the following work packages:
- Implementation of code
- Implementation of executable tests that must pass before delivery, with test coverage >= 80%
- Creation of documentation for the code
- Creation of brand-specific cmp-config files
- Setup of a project (including asset management requirements) as a copy of the consent management platform
- Removal of netID references
- Creation of brand-specific settings and files for custom purposes/providers
- Addition of new brand-specific CSS themes (variable values, logos, etc.)
- Inclusion of the required official IAB GVL translations (ES, FR) in the weekly GVL sync
- Implementation of I18n and preparation of brand-specific data sources
- Implementation of PMC2.0 backend usage modules
- Implementation of playout logic
- Implementation of the layer initialization process (mode=default and mode=resurface)
- CDN upload and release process
- Project documentation
Project implementation:
- The deliverable should be written in Typescript and Vue.js, built with Vite and tested with Vitest.
AI Consultant - Machine Learning (m/w/d)
For an AI lab we are looking for Machine learning experts to train an AI model (Large Language Model - LLM). GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Design original computational STEM problems that simulate real scientific workflows
- Create problems that require Python programming to solve
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains and creative problem-solving approaches
- Verify solutions using Python with standard libraries (numpy, pandas, scipy, sklearn)
- Document problem statements clearly and provide verified correct answers
Freelance AI Trainer - Writers (English) (m/f/d)
An AI Lab is seeking professionals experienced in working with tests to join their innovative team as English AI Trainers. The role involves crafting and editing texts, as well as evaluating AI-generated replies to ensure quality and accuracy. This position is ideal for individuals with expertise in writing, editing, and analyzing content, particularly in the context of AI and language models. As part of a cutting-edge AI lab, you will contribute to the development and refinement of advanced AI systems.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Craft and edit high-quality texts tailored to specific requirements.
- Evaluate and analyze AI-generated replies for accuracy, relevance, and quality.
- Collaborate with teams to improve AI language models and content generation processes.
- Provide feedback and suggestions to enhance AI performance.
- Conduct research to ensure content aligns with industry standards and user expectations.
Mathematician with Python Experience (m/w/d)
For an AI lab we are looking for mathematicians with python experience to train an AI model (Large Language Model - LLM).
As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities.
Although every project is unique, you might typically:
- Design original computational mathematics problems that simulate real mathematical research workflows.
- Create problems requiring Python programming to solve (using numpy, scipy, sympy).
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks).
- Develop problems requiring non-trivial reasoning chains in areas like number theory, combinatorics, graph theory, and numerical analysis.
- Base problems on real research challenges or practical applications from mathematical practice.
- Verify solutions using Python with standard mathematical libraries.
- Document problem statements clearly and provide verified correct answers.
Support in:
- Number Theory: Prime factorization, Diophantine equations, modular arithmetic, cryptographic computations.
- Combinatorics: Enumerations, partitions, generating functions, combinatorial optimization.
- Graph Theory: Network analysis, path finding, graph coloring, spanning trees.
- Numerical Analysis: Root finding, numerical integration, differential equations, matrix computations.
- Discrete Mathematics: Recurrence relations, algorithmic complexity, discrete optimization.
- Algebra: Polynomial computations, group theory calculations, matrix decompositions.
Biologist with Python Experience (m/w/d)
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as an AI Tutor in Biology, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
ERP-Transformation Manager (m/w/d)
An established company is looking for an experienced ERP Transformation Manager to take full responsibility for planning and steering a comprehensive ERP transformation program. The project's goal is harmonizing processes, implementing a new ERP system, and meeting IFRS requirements.
The ERP Transformation Manager will analyze, redesign, and standardize the commercial core processes in civil and rail construction. This includes translating IFRS requirements into system structures and posting logic, closely coordinating with Finance, Controlling, Project Management, and IT departments.
The role includes managing the ERP rollout, including fit-gap analysis, process design, test management, and migration. In addition, a unified reporting and KPI framework for group financial statements and project management will be established. The manager will act as the central interface between operational units, Finance, management, and the group, and will set up a sustainable change and training concept for users.
- Planning and steering the ERP transformation program (IFRS transition, process harmonization, ERP rollout)
- Analyzing, redesigning, and standardizing commercial core processes
- Translating IFRS requirements into system structures and posting logic
- Managing the ERP rollout, including fit-gap analysis, process design, test management, and migration
- Building a unified reporting and KPI framework
- Stakeholder management and ensuring smooth communication
- Leading interdisciplinary project teams and managing external consultants and implementation partners
- Establishing a sustainable change and training concept
- Ensuring measurable process improvements after the ERP system goes live
Freelance Ruby Developer (m/f/d)
For an AI lab we are looking for Ruby Developer to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities.
- Code generation and code review
- Prompt evaluation and complex data annotation
- Training and evaluation of large language models
- Benchmarking and agent-based code execution in sandboxed environments
- Working across multiple programming languages (Python, JavaScript/TypeScript, Rust, SQL, etc.)
- Adapting guidelines for new domains and use cases
- Collaborating with project leads, solution engineers, and supply managers on complex or experimental projects
Project Manager Brand Guardianship (m/f/d)
The service is required as part of the Brand Image Pool Photoshoot project. The project includes:
- Managing partial tasks throughout the entire Brand Image Pool photoshoot project from January to June
- Taking on brand guardianship tasks during the pool photoshoot project period
- Specific service description without personal reference:
- Independently defining, managing and executing the project. This ranges from project management to creating roadmaps and project presentations
- Developing ideas and concepts for measures
- Active management of project risks
- Active management of project issues including professional advice on escalations
- Preparing and following up on stakeholder and steering board meetings
- Defining project scope and the overall project phases
- Transparent and appropriate information to the client regarding scope, quality, schedule, budget and status
IT Project Manager ServiceNow (Senior)
- A company in the energy and energy services sector is looking for an experienced IT project manager for a ServiceNow project.
- The project aims to lead and successfully deliver an enterprise ServiceNow project focusing on ITSM and Customer Service Management (CSM).
- The role includes planning, controlling, and ensuring a stable project flow in close collaboration with internal and external stakeholders.
- Operational & strategic service management of the ServiceNow platform
- Process ownership for ITSM and CSM (B2B & B2C)
- Process design, governance & continuous optimization
- Management of external providers and vendors
- Monitoring, KPI analysis & deriving improvements
- Ensuring stable platform operations
Freelance Cybersecurity Consultant for AI Red Teaming
For an AI lab we are looking for cybersecurity consultants to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities
- Evaluate and red team AI models and agents and machine learning systems for vulnerabilities and safety risks.
- Create offline reproducible & auto-evaluable test cases to test safety & capability of AI agents.
- Develop and implement automation scripts, custom tools, environments and test harnesses.
- Lead or contribute to security research initiatives, especially in AI safety, creating and implementing realistic and challenging attack scenarios for the model.
- Advise on cybersecurity best practices and policy implications.
New
Product Manager POS / Cash Register Systems (m/f/d)
A company is looking for an experienced product manager, ideally with a background in hospitality. The goal of the project is to develop innovative product solutions and optimize existing processes to improve the customer experience and strengthen the company’s market position.
The product manager will play a key role in developing and implementing product strategies and will work closely with internal teams and external stakeholders. Travel expenses are not covered, so the candidate should preferably be based in Berlin or be willing to cover the costs themselves.
- Leading and overseeing the entire project lifecycle in the Cloud POS area
- Developing and implementing project plans, schedules, and budgets
- Coordinating between different teams and stakeholders
- Ensuring compliance with project goals and requirements
- Identifying and managing risks and issues
- Reporting to management and other relevant parties
AI Consultant for Vibe Coding (m/w/d)
An AI Lab is looking for a AI Trainer for Vibe Coding. This role involves producing accurate, well-reasoned outputs across diverse domains, leveraging automation and AI tools. The position requires expertise in coding and optimizing Python scripts, handling large datasets, improving AI-generated content, and formatting and troubleshooting technical workflows.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Conduct advanced web research and data mining using multiple tools to locate and extract information from official sources. Use LLMs and advanced prompts to refine search strategies and validate data accuracy by cross-referencing authoritative sources.
- Perform web scraping and data extraction by navigating complex website structures and multi-level pages (regions → companies → detailed pages). Handle dynamic content, archived pages, and various HTML formats, and organize extracted data into clean, well-formatted CSV files.
- Write and optimize Python scripts for data processing and analysis using libraries such as pandas, BeautifulSoup, Selenium, and matplotlib. Transform raw data into structured formats (CSV, JSON, tables) and create visualizations when required.
- Carry out data processing and quality assurance by cleaning, validating, and structuring datasets. - - Ensure data integrity across multiple sources, apply formatting specifications, and run verification steps to maintain high output quality.
- Apply strong problem-solving and task execution skills to break down complex workflows, troubleshoot technical issues independently, and adapt quickly between different domains and task types with minimal supervision.
- Produce clear documentation and high-quality outputs that follow exact requirements for file formats, naming conventions, and data structure. Maintain reproducible workflows and well-organized code.
AI Consultants - Data Science (m/w/d)
We are seeking experienced data scientists to create computationally intensive data science problems for an advanced AI evaluation project. This is a remote, project-based opportunity for experts who can design challenging problems that require computational methods to solve and mirror the full data science lifecycle - from data acquisition and processing to statistical analysis and actionable business insights.
What You'll Do
- Design original computational data science problems that simulate real-world analytical workflows across industries (telecom, finance, government, e-commerce, healthcare)
- Create problems requiring Python programming to solve (using pandas, numpy, scipy, sklearn, statsmodels, matplotlib, seaborn)
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains in data processing, statistical analysis, feature engineering, predictive modeling, and insight extraction
- Create deterministic problems with reproducible answers - avoid stochastic elements or require fixed random seeds for exact reproducibility
- Base problems on real business challenges: customer analytics, risk assessment, fraud detection, forecasting, optimization, and operational efficiency
- Design end-to-end problems spanning the complete data science pipeline (data ingestion → cleaning → EDA → modeling → validation → deployment considerations)
- Incorporate big data processing scenarios requiring scalable computational approaches
- Verify solutions using Python with standard data science libraries and statistical methods
- Document problem statements clearly with realistic business contexts and provide verified correct answers
Freelance Biology Expert for AI Model Training (m/f/d)
An AI lab is looking for a freelance biology experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in biology (all areas) contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Evaluate AI models for biology applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Fullstack Engineer (m/f/d)
- Product and web development in the data-driven field
- Shaping the software architecture for new data products
- Collaborating in interdisciplinary teams (e.g., with data scientists and business developers)
Frontend developer to HR platform with Angular experience
Reach out to us if you are interested in working with us on the project.
Sign up to get access to more exciting projects that match your skills and preferences!
AI Agent Evaluation Analyst (m/f/d)
Industry
Information Technology (IT)
Area
Quality Assurance (QA)
Project info
- Period29.12.2025 - 25.04.2026
- Capacityfrom 95%
- Daily rate200 - 320€
- LocationAmsterdam, Netherlands
- Languages Essential:
- German(Advanced)
Desirable:- English(Advanced)
- German
- Remotefrom 95%
Description
We are looking for an Freelance Agent Evaluation Analyst to take ownership of quality, structure, and insight across the project. This role goes far beyond task-checking - it’s about critical thinking, systems-level analysis, and ensuring clarity, reliability, and consistency at scale. You’ll work as both a hands-on evaluator and an analyst, collaborating with domain experts, delivery managers, and engineers. Beyond reviewing outputs, you’ll be expected to understand the “why” behind the work, identify logical gaps or inconsistencies, and propose meaningful improvements.
This is a flexible, impact-driven role where you’ll have space to grow, contribute ideas, and help shape how evaluation and quality are scaled across the project.
This role is especially well-suited for:
Analysts, researchers, or consultants with strong structuring and reasoning skills Junior product managers or strategists curious about AI and evaluation work Smart problem-solvers (students or early-career professionals) who enjoy digging into logic, systems, and edge cases
You do not need a coding background. What matters most is curiosity, intellectual rigor, and the ability to evaluate complex setups with precision.
What you’ll be doing
- Fully own the QA pipeline for agent evaluation tasks;
- Review and validate tasks and golden paths created by scenario writers and experts;
- Spot logical inconsistencies, vague requirements, hidden risks, and unrealistic assumptions;
- Provide structured feedback and ensure quality alignment across contributors; Train, onboard, and mentor new QA team members;
- Collaborate with domain experts, delivery managers, and engineers to improve test clarity and coverage;
- Maintain and improve QA checklists, SOPs, and review guidelines;
- Contribute to test planning, prioritization, and quality benchmarks;
- Take initiative to suggest new approaches, tools, and processes that help scale validation and analysis.
Requirements
What you should know / be able to do
- Strong analytical and critical thinking skills;
- Attention to detail and reliability - your work can be trusted without double-checking;
- Experience in manual QA, scenario validation, or similar analytical work;
- Comfortable working with structured formats (JSON/YAML);
- Clear written communication and documentation skills;
- Ability to give constructive feedback and coach others;
- Capable of working with a wide range of stakeholders: from engineers to directors/VPs.
Nice to have
- Background in scenario-based testing, test design, or annotation workflows;
- Experience with AI/LLM evaluation, prompt validation, or agent behavior testing;
- Some technical independence (e.g., Python skills);
- Familiarity with MCP / tool-based task execution;
- Experience working in cross-functional teams across product, delivery, and engineering.
Who you are
- Detail-obsessed but also able to see the bigger picture;
- Proactive, independent, and take true ownership of your work;
- Strong communicator who can turn complex findings into actionable insights;
- Flexible and motivated to contribute across a variety of tasks and projects;
- Believe quality is not just checking work, but making the whole product better.