Project details
Recommended projects
Evaluation Scenario Writer (m/w/d)
We’re looking for someone who can design realistic and structured evaluation scenarios for LLM-based agents. You’ll create test cases that simulate human-performed tasks and define gold-standard behavior to compare agent actions against. You’ll work to ensure each scenario is clearly defined, well-scored, and easy to execute and reuse. You’ll need a sharp analytical mindset, attention to detail, and an interest in how AI agents make decisions.
Although every project is unique, you might typically:
- Designing structured test scenarios based on real-world tasks.
- Defining the golden path and acceptable agent behavior.
- Annotating task steps, expected outputs, and edge cases.
- Working with devs to test your scenarios and improve clarity.
- Reviewing agent outputs and adapting tests accordingly
AI Evaluation Consultant (m/w/d)
We are seeking an analytical and technically-minded professional to:
- Evaluate AI outputs and processes
- Ensure quality, accuracy, and reliability
- Identify logical errors, risks, and structural inconsistencies
- Provide actionable insights and recommendations to the team
Ideal candidates:
- Consultants, auditors, analysts, data researchers, or business/technical analysts with strong reasoning skills
- Professionals curious about AI, process improvement, and quality evaluation
- Problem-solvers who enjoy analyzing complex systems, logic, and scenarios
Key Responsibilities:
- Lead evaluation of AI outputs and related processes
- Review tasks against expected/ideal scenarios; identify gaps and risks
- Provide structured, actionable recommendations to engineers, domain experts, and managers
- Maintain and improve evaluation guidelines, checklists, SOPs
- Suggest new approaches, tools, and processes to enhance AI evaluation
Software Test Engineer (m/f/d)
Your tasks:
- Specification and implementation of test cases, scenarios and plans
- Test automation using existing and new technologies
- Execution and maintenance of simulation- and hardware-based tests as well as regression tests
- Analysis of software bug reports and their verification
- Creation of test reports and documentation
- Presentation of current test results
- Taking responsibility for tests within the development or system team
- Further development and maintenance of internally used test applications
New
Vibe Coding Web Scraping Expert (m/f/d)
As a Vibe Code specialist, you'll handle data scraping tasks requiring technical precision for web extraction and processing, utilizing various tools such as our provided Apify and OpenRouter alongside your own resourceful approaches.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key Responsibilities:
- Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets.
- Leverage internal tools (Apify, OpenRouter) alongside custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements.
- Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior.
- Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery.
- Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes.
System Engineer Functional Safety (m/f/d) / Functional Safety
Your tasks:
- Safety responsibility in the development process (Functional Safety Engineering / Functional Safety Management)
- Analysis and verification of system requirements and system interface definitions
- Requirements engineering along the development process
- Verification and confirmation reviews across the entire safety lifecycle
- Coordination and alignment of work with specialist departments and external contacts
- Conducting requirements workshops, GuR, FTA, FMEA and FMECA
- Supporting system development up to market launch/approval
Freelance Automotive Engineer (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving rapidly, and our goal is to help them handle specialized questions and complex reasoning skills.
Although each project is different, your tasks may include:
- Content Creation & Refinement: Draft and polish content to ensure accuracy and relevance across various physics topics, and create task examples and references.
- Expert Evaluation: Review qualification tests of experts to confirm their skills.
- Chat Moderation: Answer project-related questions from other experts in Discord, especially about project guidelines.
- Work Auditing: Check and assess tasks done by other experts, ensure they follow guidelines, give constructive feedback, verify technical info, and edit content to improve quality.
Senior Project Manager Customer Interaction
A company is looking for support within the project to evaluate, implement and further develop quality surveys in digital channels. The goal of the project is to increase customer satisfaction in digital channels, evaluate, implement and further develop survey methods to enable a consistent collection of customer satisfaction across all channels. Improvement potentials are to be identified and implemented.
The role includes consulting, developing and implementing measures to collect and improve customer satisfaction in digital channels. Main tasks:
- Advising on survey methods to capture customer experience and quality in digital channels, market standards, benchmarks and future orientation.
- Developing a future model for quality in digital channels, relevant KPIs and survey methods as well as standard processes.
- Implementing the decided measures, including interface management and coordination with technology partners and social partners.
- Testing implemented measures to collect data and ensure the required criteria.
- Consolidating and listing existing and missing customer survey methods/quality KPIs in all responsible digital channels.
- Advising on the preparation of decision-making documents and implementing the measures required for this.
- Identifying improvement potentials and developing a standard process for transparency and implementation.
ERP Transformation Manager (m/f/d)
A company is looking for an experienced ERP Transformation Manager who will take full responsibility for planning and managing a comprehensive ERP transformation program. The project aims to harmonize processes, introduce a new ERP system, and implement IFRS requirements.
The ERP Transformation Manager will analyze, redesign, and standardize the financial core processes in civil and track construction. This includes translating IFRS requirements into system-side structures and posting logics, closely aligned with Finance, Controlling, Project Management, and IT departments.
The role covers managing the ERP implementation, including fit-gap analyses, process design, test management, and migration. A unified reporting and KPI framework for consolidated financial statements and project control will also be built. The manager will act as the central interface between operational units, finance, executive management, and the group, and establish a sustainable change and training concept for users.
- Planning and management of the ERP transformation program (IFRS transition, process harmonization, ERP rollout)
- Analysis, redesign, and standardization of financial core processes
- Translation of IFRS requirements into system-side structures and posting logics
- Management of ERP implementation including fit-gap analyses, process design, test management, and migration
- Development of a unified reporting and KPI framework
- Stakeholder management and ensuring smooth communication
- Leading interdisciplinary project teams and managing external consultants and implementation partners
- Establishing a sustainable change and training concept
- Ensuring measurable process improvements after ERP system go-live
Freelance Product Owner for Point of Sale App
- Vision & Strategy: You shape the roadmap for the POS system of the future and ensure that we not only meet our customers’ needs but anticipate them.
- Multi-Platform Excellence: You oversee the development of our app ecosystem with a focus on Android. You understand the challenges of different form factors (from the compact card terminal to the large stationary tablet).
- Delivery Management: You lead your cross-functional development team with clarity and enthusiasm. You facilitate ceremonies, remove blockers, and celebrate releases.
- Requirements Engineering: You translate complex business logic into precise technical requirements and user stories that leave developers with no questions.
Freelance Mechanical Engineer with Python Experience (m/f/d)
For an AI lab, we are looking for a Mechanical Engineer with Python experience to train a large language model (LLM).
GenAI models are improving rapidly, and one of our goals is to make them able to handle specialized questions and perform complex reasoning.
If you join us, you’ll have the chance to work on these projects. While every project is different, your tasks might include:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across various topics in Mechanical Engineering, and develop reference materials and task examples.
- Expert Screening: Evaluate experts’ qualification tests to confirm their skills.
- Chat Moderation: Support project-related questions from other experts in Discord chats, especially those about project guidelines.
- Work Auditing: Review and assess tasks completed by experts to ensure they meet project standards. Offer feedback, verify technical details, and edit content to improve quality.
Freelance Cybersecurity Consultant for AI Red Teaming
For an AI lab we are looking for cybersecurity consultants to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities
- Evaluate and red team AI models and agents and machine learning systems for vulnerabilities and safety risks.
- Create offline reproducible & auto-evaluable test cases to test safety & capability of AI agents.
- Develop and implement automation scripts, custom tools, environments and test harnesses.
- Lead or contribute to security research initiatives, especially in AI safety, creating and implementing realistic and challenging attack scenarios for the model.
- Advise on cybersecurity best practices and policy implications.
Commissioning & Qualification (C&Q) Engineer (m/w/d)
A company is looking for an experienced Commissioning & Qualification (C&Q) Engineer to qualify and commission production equipment according to GMP standards. The goal of the project is to ensure the technical and organizational prerequisites for GMP-compliant qualification of the production equipment.
- Independently carry out commissioning and qualification activities, especially in IOQ
- Operate PCS7 systems
- Work with single-use equipment
- Carrying out commissioning and qualification activities for production equipment
- Ensuring all technical and organizational prerequisites for C&Q
- GMP-compliant qualification of the associated production equipment
Senior Factor 10 Developer (IPS / IPM) (m/f/d)
An insurance company in Nuremberg is looking for a Senior Factor 10 Developer with expertise in IPS and IPM. The project covers the development and optimization of software solutions in the insurance sector, focusing on high performance and reliability. The role requires solid knowledge of Factor 10 and its applications in the insurance industry.
Main tasks:
- Develop and optimize applications using Factor 10, especially in IPS and IPM.
- Collaborate with cross-functional teams to ensure seamless integration and functionality.
- Analyze and resolve complex technical issues.
- Provide technical guidance and mentoring to junior developers.
- Ensure compliance with industry standards and best practices.
AI Consultants - Data Science (m/w/d)
We are seeking experienced data scientists to create computationally intensive data science problems for an advanced AI evaluation project. This is a remote, project-based opportunity for experts who can design challenging problems that require computational methods to solve and mirror the full data science lifecycle - from data acquisition and processing to statistical analysis and actionable business insights.
What You'll Do
- Design original computational data science problems that simulate real-world analytical workflows across industries (telecom, finance, government, e-commerce, healthcare)
- Create problems requiring Python programming to solve (using pandas, numpy, scipy, sklearn, statsmodels, matplotlib, seaborn)
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains in data processing, statistical analysis, feature engineering, predictive modeling, and insight extraction
- Create deterministic problems with reproducible answers - avoid stochastic elements or require fixed random seeds for exact reproducibility
- Base problems on real business challenges: customer analytics, risk assessment, fraud detection, forecasting, optimization, and operational efficiency
- Design end-to-end problems spanning the complete data science pipeline (data ingestion → cleaning → EDA → modeling → validation → deployment considerations)
- Incorporate big data processing scenarios requiring scalable computational approaches
- Verify solutions using Python with standard data science libraries and statistical methods
- Document problem statements clearly with realistic business contexts and provide verified correct answers
IT Project Manager ISO 27.001 - Gap Closure (m/f/d)
A company in the automotive supply industry is looking for support in the area of cyber security. The project's goal is to close gaps as part of the ISO 27001 certification. The IT project manager will play a central role in controlling and monitoring the gap closure measures.
- Controlling and monitoring the measures for gap closure.
- Consistently tracking tasks, deadlines, and responsibilities.
- Coordinating between IT, business units, information security, and, if needed, external service providers.
- Ensuring that measures are implemented ISO-27001-compliant, auditable, and documented.
- Providing transparent status reports to program management and stakeholders.
- Supporting audit preparation (evidence, action status, maturity level).
Interim Staff Product Manager (m/w/d)
A company is seeking a Staff Product Manager to lead the realization of design visions for consumer-facing, multimodal experiences across voice, chat, and beyond. The role involves bringing the product vision to life within the product team, ensuring clarity and usability across various interfaces.
The successful candidate will collaborate closely with product managers, engineers, and AI specialists to align on priorities, user needs, technical feasibility, and business impact. The focus will be on balancing simplicity and delight, ensuring interfaces feel natural and human-like across voice, chat, and graphical touchpoints.
Key responsibilities:
- Drive the realization of the design vision for multimodal experiences.
- Partner with cross-functional teams to align on priorities and user needs.
- Prototype and test interaction models for intuitive and trustworthy AI agents.
- Take end-to-end ownership of design within the product team.
- Deliver high-quality, hands-on work to shape product direction.
AI Consultant - Machine Learning (m/w/d)
For an AI lab we are looking for Machine learning experts to train an AI model (Large Language Model - LLM). GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Design original computational STEM problems that simulate real scientific workflows
- Create problems that require Python programming to solve
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains and creative problem-solving approaches
- Verify solutions using Python with standard libraries (numpy, pandas, scipy, sklearn)
- Document problem statements clearly and provide verified correct answers
Management Consultant (Senior Level) (m/f/d)
A company is looking for support in the "Crisis Management Enhancement" project. The aim of the project is to drive forward the development of crisis management (CM) and complete the implementation of the CM lifecycle. The CM lifecycle includes regular reviews of the crisis management plan, preparation and execution of crisis management exercises and workshops, as well as analysis and improvement of the crisis management system and crisis risks.
The role involves coordinating all steps of the CM lifecycle, objectively evaluating the crisis management system, and developing proposals for improvement. Company-specific templates and presentations are also developed, and tests and exercises are prepared and followed up. Documenting and tracking lessons learned are part of the tasks, along with advising on and implementing potential improvements.
HSE Specialist – Cell Manufacturing
A company in the automotive and robotics industry is looking for an experienced EHS Specialist to support cell manufacturing processes. The goal of the project is to ensure compliance with environmental, health and safety regulations and foster a safety culture within the manufacturing environment.
The role requires close collaboration with cross-functional teams to implement and maintain EHS standards, conduct risk assessments and drive continuous improvement initiatives.
Main tasks:
- Develop, implement and maintain EHS policies and procedures specifically tailored to cell manufacturing.
- Conduct regular risk assessments and audits to ensure regulatory compliance.
- Train and guide employees on EHS best practices.
- Investigate incidents and implement corrective actions to prevent recurrence.
- Work with internal teams to promote a culture of safety and sustainability.
- Monitor and report on EHS performance metrics.
Quality Compliance Auditor (GCP/GCLP/GVP) (M/W/D)
A company is looking for an experienced Quality Compliance Auditor responsible for ensuring adherence to GCP, GCLP and GVP standards. The project goal is to conduct internal and external audits, prepare and support regulatory inspections, and identify compliance gaps to derive corrective actions.
The role includes planning and conducting audits, supporting regulatory inspections, and ensuring compliance with ICH guidelines as well as EMA/FDA regulations.
- Conduct internal and external audits (GCP, GCLP, GVP)
- Prepare and support regulatory inspections (e.g. MHRA, FDA, EMA)
- Identify compliance gaps and derive corrective actions
Senior Regulatory Compliance Expert (FDA Inspection Preparation) (m/f/d)
A company is looking for a Senior Regulatory Compliance Expert to support its team in preparing for FDA inspections. The role includes conducting mock inspections, strategic consulting on inspection readiness, as well as support for pre-approvals and routine inspections. The ideal candidate has extensive expertise in regulatory compliance, especially FDA standards, and plays a key role in ensuring the company meets global compliance requirements.
- Conducting mock inspections according to FDA standards
- Strategic consulting on inspection readiness
- Support for pre-approval and routine inspections
Tax Strategy Consulting
The company is undergoing a phase with an increased need for highly specialized expertise in group taxation. The internal tax processes are to be reviewed, further developed, and realigned in selected areas. There are also requirements to optimize existing structures, review the Tax Compliance Management System, and prepare for upcoming audits and strategic decisions.
The aim of the project is to analyze, develop, and align the company’s tax-relevant structures, processes, and control mechanisms in line with legal requirements and strategic business objectives. The results should enable the company to make its tax processes more efficient and transparent and to provide a solid foundation for future tax decisions and audits.
Services
- Analysis of the existing tax processes and structures within the company, including assessment of potential optimization areas.
- Development of professional recommendations for tax strategies and for aligning relevant tax workflows within the company.
- Creation of concepts for the further development of the Tax Compliance Management System, including the identification of appropriate control mechanisms and documentation requirements.
- Preparation of professional decision-making materials for management on tax issues, especially for complex business decisions with tax implications.
- Preparation of an objective assessment for upcoming or ongoing tax audits and coordination of procedures with external consulting partners.
- Technical review of tax calculations and documents such as tax returns or year-end financial statement elements.
Java IT Architect (m/f/d)
A company is looking for an experienced Java IT Architect for a project at a development bank with a duration of five months. The project's goal is to develop and optimize the IT architecture of operational and administrative systems and to analyze and implement business requirements into the bank's IT architecture.
- Development of the IT architecture for operational and administrative systems.
- Conducting impact analyses of business requirements on the IT architecture.
- Monitoring and analyzing portfolio changes and preparing them for management decisions.
- Creating IT concepts and performing migration tasks.
- Using and evaluating AI plugins to support software development.
- Ensuring security aspects in software development (secure coding, authentication, encryption).
- Documenting the architecture and interfaces.
New
Expert in process automation for law firm environments (m/f/d)
A law firm is looking for an expert to optimize and automate processes. The goal of the project is to make workflows more efficient, especially in the areas of PDF processing, file management, and accounting processes in an SQL environment. In addition, a user-friendly user interface (UI) should be developed that is tailored to the needs of lawyers and secretaries.
- Automation of PDF processing tasks (e.g. merging, converting, editing).
- Development of solutions for automated management of file extensions and structures.
- Optimization and automation of accounting processes in an SQL environment.
- Creation of an intuitive and user-friendly UI for non-technical users such as lawyers and secretaries.
- Integration of open-source libraries to extend functionality.
- Collaboration with firm staff to gather requirements and develop solutions.
- Documentation of the developed solutions and training of users.
New
Kajabi Expert (m/f/d)
A company is looking for experienced support to build and further develop programs and content on Kajabi. The goal is to optimize the setup, structuring, and professional build-out of the content.
The role includes support for product structure, automations, and the membership area. Ideally, the freelancer brings experience in interim management, freelance experts, and independent consulting. A long-term collaboration is desired.
Safety and Health Protection Coordinator (SiGeKo) and Safety Specialist (SiFa) (m/f/d)
An organization is looking for support from a Safety and Health Protection Coordinator (SiGeKo) and Safety Specialist (SiFa) for a construction project in Hamburg. The project’s goal is to ensure compliance with all relevant safety and health regulations on the construction site and to minimize risks and hazards.
The role includes coordinating and implementing safety measures, advising site and project management, and creating and updating safety plans. The focus is on the practical application of safety concepts and continuously improving safety standards on the site.
Control System Technician / Control Systems Specialist (m/f/d)
A transformation and modernization project at the HKW Tiefstack site in Hamburg is looking for support. The goal of the project is to develop the energy and heat infrastructure into a future-proof and flexible system. Plant engineering, hydraulics, energy supply, and control technology play a central role.
The project is in the implementation phase. The previous lead will leave the project in about 2 months, so there is an immediate need to take over and ensure a smooth handover.
TM1 Planning Analytics and Interfaces Development (m/f/d)
A company is seeking support in the digital transformation of investment controlling. The role is part of a Scrum team in the finance department. The focus is on developing and optimizing TM1 Planning Analytics as the core system, including building interfaces, processing data in the model, and providing it for reporting purposes.
Cyber Security Consultant – Product Security & Regulatory Compliance (m/f/d)
A medical technology company is looking for an experienced Cyber Security Consultant to act as an independent advisor at the intersection of software architecture, DevOps and regulatory affairs. The goal of the project is to develop and secure innovative digital health products (Software as a Medical Device / Connected Devices). This position includes purely specialist consulting, technical assessments and delivery of validated security artifacts, without operational management decisions. The focus is on translating regulatory requirements (FDA, MDR, EU CRA, NIS2) into technical solutions and objectively assessing security risks.
- Holistic Threat Modeling: Conduct STRIDE-based analyses for APIs, cloud-native services, AI components and CI/CD pipelines, and maintain formal threat model reports.
- Security Architecture & Design: Create technical data flow diagrams (DFD) and security architecture review reports to document security decisions in cloud and container environments.
- DevSecOps & Supply Chain Security: Advise on hardening CI/CD pipelines, implementing "security-by-default" and managing SBOM artifacts (third-party risk).
- Penetration Testing Support: Define the scope for pen tests and fuzzing, and technically validate the results and CVSS scores.
- Vulnerability Management: Assess vulnerabilities (vulnerability impact assessments) and develop remediation plans.
- Audit Preparation: Compile technical evidence and documentation for regulatory submissions (FDA, MDR, MDS2).
Senior Cloud Developer TypeScript (m/f/d)
At our client, changes in operations support and related work in the cloud native environment are on the agenda.
- Development of the delivery pipeline
- Development of the applications catalog for ODC
- Development of the ITSM/CMDB integration
Frontend developer to HR platform with Angular experience
Reach out to us if you are interested in working with us on the project.
Sign up to get access to more exciting projects that match your skills and preferences!
Time's up! We are no longer accepting applications.
AI Agent Evaluation Analyst (m/w/d)
Industry
Information Technology (IT)
Areas
Quality Assurance (QA)
Research and Development (R&D)
Project info
- Daily ratefrom 280€
- Language
- English(Advanced)
- English
- Remote100%
Description
We’re on the hunt for QAs for autonomous AI agents for a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks. Throughout the project, you’ll have to balance quality assurance, research, and logical problem-solving. This project opportunity is ideal for people who enjoy looking at systems holistically and thinking through scenarios, implications, and edge cases.
You do not need a coding background, but you must be curious, intellectually rigorous, and capable of evaluating the soundness and consistency of complex setups. If you’ve ever excelled in things like consulting, CHGK, Olympiads, case solving, or systems thinking — you might be a great fit.
What you’ll be doing:
- Reviewing evaluation tasks and scenarios for logic, completeness, and realism.
- Identifying inconsistencies, missing assumptions, or unclear decision points.
- Helping define clear expected behaviors (gold standards) for AI agents.
- Annotating cause-effect relationships, reasoning paths, and plausible alternatives.
- Thinking through complex systems and policies as a human would to ensure agents are tested properly.
- Working closely with QA, writers, or developers to suggest refinements or edge case coverage.
Requirements
- Excellent analytical thinking: Can reason about complex systems, scenarios, and logical implications.
- Strong attention to detail: Can spot contradictions, ambiguities, and vague requirements.
- Familiarity with structured data formats: Can read, not necessarily write JSON/YAML.
- Can assess scenarios holistically: What's missing, what’s unrealistic, what might break?
- Good communication and clear writing (in English) to document your findings.
We also value applicants who have:
- Experience with policy evaluation, logic puzzles, case studies, or structured scenario design.
- Background in consulting, academia, olympiads (e.g. logic/math/informatics), or research.
- Exposure to LLMs, prompt engineering, or AI-generated content.
- Familiarity with QA or test-case thinking (edge cases, failure modes, “what could go wrong”). Some understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.).