Project details
Recommended projects
AI Agent Evaluation Analyst (m/f/d)
We are looking for a Freelance Agent Evaluation Analyst to take ownership of quality, structure, and insight across the project. This role goes far beyond task-checking - it’s about critical thinking, systems-level analysis, and ensuring clarity, reliability, and consistency at scale. You’ll work as both a hands-on evaluator and an analyst, collaborating with domain experts, delivery managers, and engineers. Beyond reviewing outputs, you’ll be expected to understand the “why” behind the work, identify logical gaps or inconsistencies, and propose meaningful improvements.
This is a flexible, impact-driven role where you’ll have space to grow, contribute ideas, and help shape how evaluation and quality are scaled across the project.
This role is especially well-suited for:
Analysts, researchers, or consultants with strong structuring and reasoning skills Junior product managers or strategists curious about AI and evaluation work Smart problem-solvers (students or early-career professionals) who enjoy digging into logic, systems, and edge cases
You do not need a coding background. What matters most is curiosity, intellectual rigor, and the ability to evaluate complex setups with precision.
What you’ll be doing
- Fully own the QA pipeline for agent evaluation tasks;
- Review and validate tasks and golden paths created by scenario writers and experts;
- Spot logical inconsistencies, vague requirements, hidden risks, and unrealistic assumptions;
- Provide structured feedback and ensure quality alignment across contributors; Train, onboard, and mentor new QA team members;
- Collaborate with domain experts, delivery managers, and engineers to improve test clarity and coverage;
- Maintain and improve QA checklists, SOPs, and review guidelines;
- Contribute to test planning, prioritization, and quality benchmarks;
- Take initiative to suggest new approaches, tools, and processes that help scale validation and analysis.
AI Agent Evaluation Analyst
For an AI lab we are looking for AI Agent Evaluation Analyst to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities
- Reviewing evaluation tasks and scenarios for logic, completeness, and realism.
- Identifying inconsistencies, missing assumptions, or unclear decision points.
- Helping define clear expected behaviors (gold standards) for AI agents.
- Annotating cause-effect relationships, reasoning paths, and plausible alternatives.
- Thinking through complex systems and policies as a human would to ensure agents are tested properly.
- Working closely with QA, writers, or developers to suggest refinements or edge case coverage.
AI Evaluation Consultant (m/w/d)
We are seeking an analytical and technically-minded professional to:
- Evaluate AI outputs and processes
- Ensure quality, accuracy, and reliability
- Identify logical errors, risks, and structural inconsistencies
- Provide actionable insights and recommendations to the team
Ideal candidates:
- Consultants, auditors, analysts, data researchers, or business/technical analysts with strong reasoning skills
- Professionals curious about AI, process improvement, and quality evaluation
- Problem-solvers who enjoy analyzing complex systems, logic, and scenarios
Key Responsibilities:
- Lead evaluation of AI outputs and related processes
- Review tasks against expected/ideal scenarios; identify gaps and risks
- Provide structured, actionable recommendations to engineers, domain experts, and managers
- Maintain and improve evaluation guidelines, checklists, SOPs
- Suggest new approaches, tools, and processes to enhance AI evaluation
Business Analyst – SAP S/4HANA Output Management (m/f/d)
- A company is looking for an experienced Business Analyst to support the transformation from SAP ECC to S/4HANA Utilities.
- The project aims to analyze, document and optimize output and archiving processes, as well as to create functional designs and specifications.
- The analyst will work closely with product owners, IT and business units to align on feasibility, effort and priority of requirements.
Freelance Automotive Engineer (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Chemistry Expert for AI Model Training (m/f/d)
An AI lab is looking for a freelance chemistry experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in chemistry contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Evaluate AI models for chemistry applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Freelance Electrical Engineer with Python Experience (m/w/d)
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Mechanical Engineer with Python Experience (m/w/d)
For an AI lab we are looking for Mechanical Engineer with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Mechanical Engineering, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Civil Engineer with Python Experience (m/f/d)
A company is looking for a freelance Civil engineering experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in civil engineering contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
Key responsibilities:
- Evaluate AI models for civil engineering applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Freelance Statistics Expert with Python Experience (m/f/d)
For an AI lab we are looking for Statistics Expert with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
Freelance Physics Expert (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Expert for Building a Call Center
A health insurance company is urgently looking for an expert to build a call center. The goal of the project is to develop a comprehensive roadmap that covers all aspects of setting up the call center. The expert will work closely with internal stakeholders to analyze requirements and develop a structured approach.
- Create a roadmap for setting up a call center
- Identify and select suitable systems, tools, and technologies
- Develop organizational and staffing structures
- Define necessary processes
- Plan realistic implementation phases
Chemist with Python Experience (m/f/d)
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as an AI Tutor in Chemistry, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
Physicist with Python Experience (m/w/d)
For an AI lab we are looking for phycists with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a phycist, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Design original computational physic problems that simulate real research workflows.
- Create problems requiring Python programming to solve (using numpy, scipy, sympy).
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks).
- Develop problems requiring non-trivial reasoning chains.
- Base problems on real research challenges or practical applications from physical practice.
- Verify solutions using Python with standard libraries.
- Document problem statements clearly and provide verified correct answers.
Project Manager Magazines / Magazine Production (m/f/d)
- Responsibility for coordinating and managing the entire production process of magazines
- Planning and monitoring issue structure, deadlines, ads, and workflows
- Close collaboration with editorial, publishing management, marketing, IT, sales, printers, and service providers
- Quality assurance of layouts, texts, and print approvals
- Cost calculation and organization of supplementary products (e.g. inserts, posters, extensions)
- Active role in strategic projects, conferences, and in launching new formats
New
AI Consultant - Machine Learning (m/w/d)
For an AI lab we are looking for Machine learning experts to train an AI model (Large Language Model - LLM). GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Design original computational STEM problems that simulate real scientific workflows
- Create problems that require Python programming to solve
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains and creative problem-solving approaches
- Verify solutions using Python with standard libraries (numpy, pandas, scipy, sklearn)
- Document problem statements clearly and provide verified correct answers
Developer for Consent Management Implementation (m/f/d)
To replace the consent layers on the web for our international brands that were previously provided by third-party CMPs, we need to reimplement these layers so they can be operated and delivered in-house. This requires solid knowledge in Typescript, Vue.js and classic web technologies (HTML and CSS). The goal is to deliver executable code that implements all requirements and includes automated tests to verify correct functionality.
What exactly is the scope of the assignment: The main focus is on developing elements for decision-making on the approach and on implementing measures along the designed project course. This specifically includes the following work packages:
- Implementation of code
- Implementation of executable tests that must pass for delivery, test coverage >= 80%
- Creation of code documentation
- Creation of brand-specific cmp-config files.
- Creation of a project (including asset management requirements) as a copy of the consent management platform.
- Removal of netID references.
- Creation of brand-specific settings and files for custom purposes/vendors.
- Adding new brand-specific CSS themes (variable values, logos, etc.).
- Incorporation of the required official IAB GVL translations (ES, FR) into the weekly synchronization with GVL
- Implementation of I18n and preparation of brand-specific data sources
- Implementation of PMC2.0 backend usage modules
- Implementation of the playout logic
- Implementation of the layer initialization process (mode=default and mode=resurface)
- CDN upload and release process
- Project documentation
Project execution:
- The desired result should be written in Typescript and Vue.js, built with Vite, tested with Vitest.
Freelance AI Trainer - Writers (English) (m/w/d)
An AI Lab is seeking professionals experienced in working with tests to join their innovative team as English AI Trainers. The role involves crafting and editing texts, as well as evaluating AI-generated replies to ensure quality and accuracy. This position is ideal for individuals with expertise in writing, editing, and analyzing content, particularly in the context of AI and language models. As part of a cutting-edge AI lab, you will contribute to the development and refinement of advanced AI systems.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Craft and edit high-quality texts tailored to specific requirements.
- Evaluate and analyze AI-generated replies for accuracy, relevance, and quality.
- Collaborate with teams to improve AI language models and content generation processes.
- Provide feedback and suggestions to enhance AI performance.
- Conduct research to ensure content aligns with industry standards and user expectations.
Mathematician with Python Experience (m/w/d)
For an AI lab we are looking for mathematicians with python experience to train an AI model (Large Language Model - LLM).
As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities.
Although every project is unique, you might typically:
- Design original computational mathematics problems that simulate real mathematical research workflows.
- Create problems requiring Python programming to solve (using numpy, scipy, sympy).
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks).
- Develop problems requiring non-trivial reasoning chains in areas like number theory, combinatorics, graph theory, and numerical analysis.
- Base problems on real research challenges or practical applications from mathematical practice.
- Verify solutions using Python with standard mathematical libraries.
- Document problem statements clearly and provide verified correct answers.
Support in:
- Number Theory: Prime factorization, Diophantine equations, modular arithmetic, cryptographic computations.
- Combinatorics: Enumerations, partitions, generating functions, combinatorial optimization.
- Graph Theory: Network analysis, path finding, graph coloring, spanning trees.
- Numerical Analysis: Root finding, numerical integration, differential equations, matrix computations.
- Discrete Mathematics: Recurrence relations, algorithmic complexity, discrete optimization.
- Algebra: Polynomial computations, group theory calculations, matrix decompositions.
Biologist with Python Experience (m/w/d)
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as an AI Tutor in Biology, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
ERP-Transformation Manager (m/w/d)
An established company is looking for an experienced ERP Transformation Manager to take full responsibility for planning and steering a comprehensive ERP transformation program. The project's goal is harmonizing processes, implementing a new ERP system, and meeting IFRS requirements.
The ERP Transformation Manager will analyze, redesign, and standardize the commercial core processes in civil and rail construction. This includes translating IFRS requirements into system structures and posting logic, closely coordinating with Finance, Controlling, Project Management, and IT departments.
The role includes managing the ERP rollout, including fit-gap analysis, process design, test management, and migration. In addition, a unified reporting and KPI framework for group financial statements and project management will be established. The manager will act as the central interface between operational units, Finance, management, and the group, and will set up a sustainable change and training concept for users.
- Planning and steering the ERP transformation program (IFRS transition, process harmonization, ERP rollout)
- Analyzing, redesigning, and standardizing commercial core processes
- Translating IFRS requirements into system structures and posting logic
- Managing the ERP rollout, including fit-gap analysis, process design, test management, and migration
- Building a unified reporting and KPI framework
- Stakeholder management and ensuring smooth communication
- Leading interdisciplinary project teams and managing external consultants and implementation partners
- Establishing a sustainable change and training concept
- Ensuring measurable process improvements after the ERP system goes live
Frontend developer to HR platform with Angular experience
Reach out to us if you are interested in working with us on the project.
Sign up to get access to more exciting projects that match your skills and preferences!
AI Agent Evaluation Analyst (m/f/d)
Industry
Information Technology (IT)
Area
Quality Assurance (QA)
Project info
- Period22.12.2025 - 18.04.2026
- Capacityfrom 95%
- Daily rate200 - 320€
- LocationAmsterdam, Netherlands
- Languages Essential:
- German(Advanced)
Desirable:- English(Advanced)
- German
- Remotefrom 95%
Description
We are looking for a Freelance Agent Evaluation Analyst to take ownership of quality, structure, and insight across the project. This role goes far beyond task-checking - it’s about critical thinking, systems-level analysis, and ensuring clarity, reliability, and consistency at scale. You’ll work as both a hands-on evaluator and an analyst, collaborating with domain experts, delivery managers, and engineers. Beyond reviewing outputs, you’ll be expected to understand the “why” behind the work, identify logical gaps or inconsistencies, and propose meaningful improvements.
This is a flexible, impact-driven role where you’ll have space to grow, contribute ideas, and help shape how evaluation and quality are scaled across the project.
This role is especially well-suited for:
Analysts, researchers, or consultants with strong structuring and reasoning skills Junior product managers or strategists curious about AI and evaluation work Smart problem-solvers (students or early-career professionals) who enjoy digging into logic, systems, and edge cases
You do not need a coding background. What matters most is curiosity, intellectual rigor, and the ability to evaluate complex setups with precision.
What you’ll be doing
- Fully own the QA pipeline for agent evaluation tasks;
- Review and validate tasks and golden paths created by scenario writers and experts;
- Spot logical inconsistencies, vague requirements, hidden risks, and unrealistic assumptions;
- Provide structured feedback and ensure quality alignment across contributors; Train, onboard, and mentor new QA team members;
- Collaborate with domain experts, delivery managers, and engineers to improve test clarity and coverage;
- Maintain and improve QA checklists, SOPs, and review guidelines;
- Contribute to test planning, prioritization, and quality benchmarks;
- Take initiative to suggest new approaches, tools, and processes that help scale validation and analysis.
Requirements
What you should know / be able to do
- Strong analytical and critical thinking skills;
- Attention to detail and reliability - your work can be trusted without double-checking;
- Experience in manual QA, scenario validation, or similar analytical work;
- Comfortable working with structured formats (JSON/YAML);
- Clear written communication and documentation skills;
- Ability to give constructive feedback and coach others;
- Capable of working with a wide range of stakeholders: from engineers to directors/VPs.
Nice to have
- Background in scenario-based testing, test design, or annotation workflows;
- Experience with AI/LLM evaluation, prompt validation, or agent behavior testing;
- Some technical independence (e.g., Python skills);
- Familiarity with MCP / tool-based task execution;
- Experience working in cross-functional teams across product, delivery, and engineering.
Who you are
- Detail-obsessed but also able to see the bigger picture;
- Proactive, independent, and take true ownership of your work;
- Strong communicator who can turn complex findings into actionable insights;
- Flexible and motivated to contribute across a variety of tasks and projects;
- Believe quality is not just checking work, but making the whole product better.
English
Powered by FRATCH.IO GmbH