We are looking for an Freelance Agent Evaluation Analyst to take ownership of quality, structure, and insight across the project. This role goes far beyond task-checking - it’s about critical thinking, systems-level analysis, and ensuring clarity, reliability, and consistency at scale. You’ll work as both a hands-on evaluator and an analyst, collaborating with domain experts, delivery managers, and engineers. Beyond reviewing outputs, you’ll be expected to understand the “why” behind the work, identify logical gaps or inconsistencies, and propose meaningful improvements. This is a flexible, impact-driven role where you’ll have space to grow, contribute ideas, and help shape how evaluation and quality are scaled across the project. This role is especially well-suited for: Analysts, researchers, or consultants with strong structuring and reasoning skills Junior product managers or strategists curious about AI and evaluation work Smart problem-solvers (students or early-career professionals) who enjoy digging into logic, systems, and edge cases You do not need a coding background. What matters most is curiosity, intellectual rigor, and the ability to evaluate complex setups with precision. What you’ll be doing - Fully own the QA pipeline for agent evaluation tasks; - Review and validate tasks and golden paths created by scenario writers and experts; - Spot logical inconsistencies, vague requirements, hidden risks, and unrealistic assumptions; - Provide structured feedback and ensure quality alignment across contributors; Train, onboard, and mentor new QA team members; - Collaborate with domain experts, delivery managers, and engineers to improve test clarity and coverage; - Maintain and improve QA checklists, SOPs, and review guidelines; - Contribute to test planning, prioritization, and quality benchmarks; - Take initiative to suggest new approaches, tools, and processes that help scale validation and analysis.

For an AI lab we are looking for Java Developer to train an AI model (Large Language Model - LLM). You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities. - Code generation and code review - Prompt evaluation and complex data annotation - Training and evaluation of large language models - Benchmarking and agent-based code execution in sandboxed environments - Working across multiple programming languages - Adapting guidelines for new domains and use cases - Following project-specific rubrics and requirements - Collaborating with project leads, solution engineers, and supply managers on complex or experimental projects

An AI Lab is looking for an AI Trainer for Vibe Coding. This role involves producing accurate, well-reasoned outputs across diverse domains, leveraging automation and AI tools. The position requires expertise in coding and optimizing Python scripts, handling large datasets, improving AI-generated content, and formatting and troubleshooting technical workflows. This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time. Key responsibilities: - Develop and optimize Python scripts for automation and AI tasks. - Handle and analyze large datasets efficiently. - Improve and refine AI-generated content for accuracy and quality. - Format and troubleshoot technical workflows to ensure smooth operations. - Collaborate with cross-functional teams to enhance AI tools and processes.

For an AI lab we are looking for Ruby Developer to train an AI model (Large Language Model - LLM). You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities. - Code generation and code review - Prompt evaluation and complex data annotation - Training and evaluation of large language models - Benchmarking and agent-based code execution in sandboxed environments - Working across multiple programming languages (Python, JavaScript/TypeScript, Rust, SQL, etc.) - Adapting guidelines for new domains and use cases - Collaborating with project leads, solution engineers, and supply managers on complex or experimental projects

For replacing the previously displayed consent layers by third-party CMPs on the web for our international brands, these layers will be reimplemented so they can be maintained and deployed in-house. This requires solid knowledge of Typescript, Vue.js, and classic web rendering technologies (HTML and CSS). The goal is to deliver executable code that implements all requirements and includes automated tests proving correct functionality. What exactly is the scope of the assignment: The main focus is on developing elements for a decision-making template on the approach and on implementing measures along the designed project course. Concretely, this includes the following work packages: - Implementation of code - Implementation of executable tests, which must pass for delivery, with test coverage >= 80% - Creation of documentation for the code - Creation of brand-specific cmp-config files. - Creation of a project (including asset management requirements) as a copy of the consent management platform. - Removal of netID references. - Creation of brand-specific settings and files for custom purposes/vendors. - Adding new brand-specific CSS themes (variable values, logos, etc.). - Inclusion of required official IAB GVL translations (ES, FR) in the weekly GVL synchronization. - Implementation of I18n and preparation of brand-specific data sources. - Implementation of PMC2.0 backend usage modules. - Implementation of playout logic. - Implementation of the layer initialization process (mode=default and mode=resurface). - CDN upload and release process. - Project documentation Project implementation: - The desired result should be written in Typescript and Vue.js, build via Vite, tests via Vitest.

Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. Although every project is unique, you might typically: - Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks. - Experts Acquisition: Assess the qualification tests of experts, ensuring their competency. - Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines. - Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.

Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. Although every project is unique, you might typically: - Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks. - Experts Acquisition: Assess the qualification tests of experts, ensuring their competency. - Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines. - Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.

Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. Although every project is unique, you might typically: - Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks. - Experts Acquisition: Assess the qualification tests of experts, ensuring their competency. - Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines. - Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.

For an AI lab we are looking for a Rust Developer to train an AI model (Large Language Model (LLM)). You help AI to make sense of the world. As a consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (at least a few hours per week) and those interested in full-time opportunities. - Code generation and code review - Prompt evaluation and complex data annotation - Training and evaluation of large language models - Benchmarking and agent-based code execution in sandboxed environments - Working across multiple programming languages (Python, JavaScript/TypeScript, Rust, SQL, etc.) - Adapting guidelines for new domains and use cases - Following project-specific rubrics and requirements - Collaborating with project leads, solution engineers, and supply managers on complex or experimental projects

- Product and web development in the data-driven area - Shaping the software architecture for new data products - Collaborating in interdisciplinary teams (e.g. with data scientists and business developers)

For an AI lab we are looking for AI Agent Evaluation Analyst to train an AI model (Large Language Model - LLM). You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities - Reviewing evaluation tasks and scenarios for logic, completeness, and realism. - Identifying inconsistencies, missing assumptions, or unclear decision points. - Helping define clear expected behaviors (gold standards) for AI agents. - Annotating cause-effect relationships, reasoning paths, and plausible alternatives. - Thinking through complex systems and policies as a human would to ensure agents are tested properly. - Working closely with QA, writers, or developers to suggest refinements or edge case coverage.

For an AI lab we are looking for physicists with Python experience to train an AI model (Large Language Model - LLM). GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a physicist, you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically: - Design original computational physics problems that simulate real research workflows. - Create problems requiring Python programming to solve (using numpy, scipy, sympy). - Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks). - Develop problems requiring non-trivial reasoning chains. - Base problems on real research challenges or practical applications from physical practice. - Verify solutions using Python with standard libraries. - Document problem statements clearly and provide verified correct answers.

For an AI lab we are looking for mathematicians with python experience to train an AI model (Large Language Model - LLM). As consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities. Although every project is unique, you might typically: - Design original computational mathematics problems that simulate real mathematical research workflows. - Create problems requiring Python programming to solve (using numpy, scipy, sympy). - Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks). - Develop problems requiring non-trivial reasoning chains in areas like number theory, combinatorics, graph theory, and numerical analysis. - Base problems on real research challenges or practical applications from mathematical practice. - Verify solutions using Python with standard mathematical libraries. - Document problem statements clearly and provide verified correct answers. Support in: - Number Theory: Prime factorization, Diophantine equations, modular arithmetic, cryptographic computations. - Combinatorics: Enumerations, partitions, generating functions, combinatorial optimization. - Graph Theory: Network analysis, path finding, graph coloring, spanning trees. - Numerical Analysis: Root finding, numerical integration, differential equations, matrix computations. - Discrete Mathematics: Recurrence relations, algorithmic complexity, discrete optimization. - Algebra: Polynomial computations, group theory calculations, matrix decompositions.

- You develop modern, high-performance web frontends with React, TypeScript, HTML, and CSS - You implement responsive designs with a focus on accessibility and performance - You plan and run unit and integration tests (for example with Playwright) - Troubleshooting in development, test, or live environments

An AI lab is looking for a freelance mathematics experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in mathematics contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights. This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time. Key responsibilities: - Evaluate AI models for mathematics applications. - Analyze model outputs and provide feedback for improvement. - Collaborate with the development team to ensure alignment with industry standards. - Document findings and recommendations for model optimization. - Conduct tests to validate model performance and reliability.

An AI lab is looking for a freelance chemistry experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in chemistry contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights. This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time. Key responsibilities: - Evaluate AI models for chemistry applications. - Analyze model outputs and provide feedback for improvement. - Collaborate with the development team to ensure alignment with industry standards. - Document findings and recommendations for model optimization. - Conduct tests to validate model performance and reliability.

An AI lab is looking for a freelance physics experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in physics contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights. This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time. Key responsibilities: - Evaluate AI models for physics applications. - Analyze model outputs and provide feedback for improvement. - Collaborate with the development team to ensure alignment with industry standards. - Document findings and recommendations for model optimization. - Conduct tests to validate model performance and reliability.

For our client, we're looking for a German-speaking AI consultant: As a consultant, you might be invited to take part in online projects to train models in your area of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum a few hours/week) and those interested in full-time opportunities. Responsibilities: - Carefully review provided data (text, images, or videos). - Review tasks submitted by the development team and ensure quality assurance/quality control. - Label or classify content based on project guidelines. - Identify and flag factually incorrect, sensitive, inappropriate, or unclear material.

For our client we are looking for a Japanese speaking AI consultant: As a consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum of a few hours per week) and those interested in full-time opportunities. Responsibilities: - Carefully review provided data (text, images, or videos). - Review tasks submitted by the developer team and ensure quality assurance/quality control. - Label or classify content based on project guidelines. - Identify and flag factually incorrect, sensitive, inappropriate, or unclear material.

For an AI lab we are looking for cybersecurity consultants to train an AI model (Large Language Model - LLM). You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise. This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities - Evaluate and red team AI models and agents and machine learning systems for vulnerabilities and safety risks. - Create offline reproducible & auto-evaluable test cases to test safety & capability of AI agents. - Develop and implement automation scripts, custom tools, environments and test harnesses. - Lead or contribute to security research initiatives, especially in AI safety, creating and implementing realistic and challenging attack scenarios for the model. - Advise on cybersecurity best practices and policy implications.