Project details
Recommended projects
Evaluation Scenario Writer (m/w/d)
We’re looking for someone who can design realistic and structured evaluation scenarios for LLM-based agents. You’ll create test cases that simulate human-performed tasks and define gold-standard behavior to compare agent actions against. You’ll work to ensure each scenario is clearly defined, well-scored, and easy to execute and reuse. You’ll need a sharp analytical mindset, attention to detail, and an interest in how AI agents make decisions.
Although every project is unique, you might typically:
- Designing structured test scenarios based on real-world tasks.
- Defining the golden path and acceptable agent behavior.
- Annotating task steps, expected outputs, and edge cases.
- Working with devs to test your scenarios and improve clarity.
- Reviewing agent outputs and adapting tests accordingly
AI Evaluation Consultant (m/w/d)
We are seeking an analytical and technically-minded professional to:
- Evaluate AI outputs and processes
- Ensure quality, accuracy, and reliability
- Identify logical errors, risks, and structural inconsistencies
- Provide actionable insights and recommendations to the team
Ideal candidates:
- Consultants, auditors, analysts, data researchers, or business/technical analysts with strong reasoning skills
- Professionals curious about AI, process improvement, and quality evaluation
- Problem-solvers who enjoy analyzing complex systems, logic, and scenarios
Key Responsibilities:
- Lead evaluation of AI outputs and related processes
- Review tasks against expected/ideal scenarios; identify gaps and risks
- Provide structured, actionable recommendations to engineers, domain experts, and managers
- Maintain and improve evaluation guidelines, checklists, SOPs
- Suggest new approaches, tools, and processes to enhance AI evaluation
Freelance Electrical Engineer with Python Experience (m/w/d)
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Automotive Engineer (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Mechanical Engineer with Python Experience (m/w/d)
For an AI lab we are looking for Mechanical Engineer with python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Mechanical Engineering, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
Freelance Ruby Developer (m/f/d)
For an AI lab we are looking for Ruby Developer to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities.
- Code generation and code review
- Prompt evaluation and complex data annotation
- Training and evaluation of large language models
- Benchmarking and agent-based code execution in sandboxed environments
- Working across multiple programming languages (Python, JavaScript/TypeScript, Rust, SQL, etc.)
- Adapting guidelines for new domains and use cases
- Collaborating with project leads, solution engineers, and supply managers on complex or experimental projects
Freelance Chemistry Expert for AI Model Training (m/f/d)
An AI lab is looking for a freelance chemistry experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in chemistry contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Evaluate AI models for chemistry applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Freelance Physics Expert (with Python) - Quality Assurance / AI Trainer
Generative AI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills.
Although every project is unique, you might typically:
- Content Creation & Refinement: Create and refine content to ensure accuracy and relevance across a variety of topics in Physics, while also developing references and examples of tasks.
- Experts Acquisition: Assess the qualification tests of experts, ensuring their competency.
- Chat Moderation: Provide support by addressing project-related questions from other experts in Discord chats, especially those related to project guidelines.
- Auditing Work: Review and evaluate tasks completed by other experts, ensuring they align with project guidelines. Provide constructive feedback, verify expertise-related information, and edit content as necessary to improve quality.
AI Consultant - Machine Learning (m/w/d)
For an AI lab we are looking for Machine learning experts to train an AI model (Large Language Model - LLM). GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join you’ll have the opportunity to collaborate on these projects. Although every project is unique, you might typically:
- Design original computational STEM problems that simulate real scientific workflows
- Create problems that require Python programming to solve
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains and creative problem-solving approaches
- Verify solutions using Python with standard libraries (numpy, pandas, scipy, sklearn)
- Document problem statements clearly and provide verified correct answers
AI Consultant for Vibe Coding (m/w/d)
An AI Lab is looking for a AI Trainer for Vibe Coding. This role involves producing accurate, well-reasoned outputs across diverse domains, leveraging automation and AI tools. The position requires expertise in coding and optimizing Python scripts, handling large datasets, improving AI-generated content, and formatting and troubleshooting technical workflows.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Conduct advanced web research and data mining using multiple tools to locate and extract information from official sources. Use LLMs and advanced prompts to refine search strategies and validate data accuracy by cross-referencing authoritative sources.
- Perform web scraping and data extraction by navigating complex website structures and multi-level pages (regions → companies → detailed pages). Handle dynamic content, archived pages, and various HTML formats, and organize extracted data into clean, well-formatted CSV files.
- Write and optimize Python scripts for data processing and analysis using libraries such as pandas, BeautifulSoup, Selenium, and matplotlib. Transform raw data into structured formats (CSV, JSON, tables) and create visualizations when required.
- Carry out data processing and quality assurance by cleaning, validating, and structuring datasets. - - Ensure data integrity across multiple sources, apply formatting specifications, and run verification steps to maintain high output quality.
- Apply strong problem-solving and task execution skills to break down complex workflows, troubleshoot technical issues independently, and adapt quickly between different domains and task types with minimal supervision.
- Produce clear documentation and high-quality outputs that follow exact requirements for file formats, naming conventions, and data structure. Maintain reproducible workflows and well-organized code.
Freelance Biology Expert for AI Model Training (m/f/d)
An AI lab is looking for a freelance biology experts to evaluate AI models. The goal of the project is to assess the performance, accuracy, and reliability of AI models applied in biology (all areas) contexts. The role involves working closely with the development team to ensure the models meet industry standards and provide actionable insights.
This is a remote part-time role that can be flexibly tailored to your availability – from just a few hours per week to full-time.
Key responsibilities:
- Evaluate AI models for biology applications.
- Analyze model outputs and provide feedback for improvement.
- Collaborate with the development team to ensure alignment with industry standards.
- Document findings and recommendations for model optimization.
- Conduct tests to validate model performance and reliability.
Freelance Java Developer (m/w/d)
For an AI lab we are looking for Java Developer to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities.
- Code generation and code review
- Prompt evaluation and complex data annotation
- Training and evaluation of large language models
- Benchmarking and agent-based code execution in sandboxed environments
- Working across multiple programming languages
- Adapting guidelines for new domains and use cases
- Following project-specific rubrics and requirements
- Collaborating with project leads, solution engineers, and supply managers on complex or experimental projects
Physicist with Python Experience (m/w/d)
For an AI lab we are looking for physicists with Python experience to train an AI model (Large Language Model - LLM).
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as a physicist, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Design original computational physics problems that simulate real research workflows.
- Create problems requiring Python programming to solve (using numpy, scipy, sympy).
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks).
- Develop problems requiring non-trivial reasoning chains.
- Base problems on real research challenges or practical applications from physical practice.
- Verify solutions using Python with standard libraries.
- Document problem statements clearly and provide verified correct answers.
Mathematician with Python Experience (m/w/d)
For an AI lab we are looking for mathematicians with python experience to train an AI model (Large Language Model - LLM).
As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities.
Although every project is unique, you might typically:
- Design original computational mathematics problems that simulate real mathematical research workflows.
- Create problems requiring Python programming to solve (using numpy, scipy, sympy).
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks).
- Develop problems requiring non-trivial reasoning chains in areas like number theory, combinatorics, graph theory, and numerical analysis.
- Base problems on real research challenges or practical applications from mathematical practice.
- Verify solutions using Python with standard mathematical libraries.
- Document problem statements clearly and provide verified correct answers.
Support in:
- Number Theory: Prime factorization, Diophantine equations, modular arithmetic, cryptographic computations.
- Combinatorics: Enumerations, partitions, generating functions, combinatorial optimization.
- Graph Theory: Network analysis, path finding, graph coloring, spanning trees.
- Numerical Analysis: Root finding, numerical integration, differential equations, matrix computations.
- Discrete Mathematics: Recurrence relations, algorithmic complexity, discrete optimization.
- Algebra: Polynomial computations, group theory calculations, matrix decompositions.
Freelance Cybersecurity Consultant for AI Red Teaming
For an AI lab we are looking for cybersecurity consultants to train an AI model (Large Language Model - LLM).
You help AI to make sense of the world. As consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum few hours/week) and those interested in full-time opportunities
- Evaluate and red team AI models and agents and machine learning systems for vulnerabilities and safety risks.
- Create offline reproducible & auto-evaluable test cases to test safety & capability of AI agents.
- Develop and implement automation scripts, custom tools, environments and test harnesses.
- Lead or contribute to security research initiatives, especially in AI safety, creating and implementing realistic and challenging attack scenarios for the model.
- Advise on cybersecurity best practices and policy implications.
AI Consultants - Data Science (m/w/d)
We are seeking experienced data scientists to create computationally intensive data science problems for an advanced AI evaluation project. This is a remote, project-based opportunity for experts who can design challenging problems that require computational methods to solve and mirror the full data science lifecycle - from data acquisition and processing to statistical analysis and actionable business insights.
What You'll Do
- Design original computational data science problems that simulate real-world analytical workflows across industries (telecom, finance, government, e-commerce, healthcare)
- Create problems requiring Python programming to solve (using pandas, numpy, scipy, sklearn, statsmodels, matplotlib, seaborn)
- Ensure problems are computationally intensive and cannot be solved manually within reasonable timeframes (days/weeks)
- Develop problems requiring non-trivial reasoning chains in data processing, statistical analysis, feature engineering, predictive modeling, and insight extraction
- Create deterministic problems with reproducible answers - avoid stochastic elements or require fixed random seeds for exact reproducibility
- Base problems on real business challenges: customer analytics, risk assessment, fraud detection, forecasting, optimization, and operational efficiency
- Design end-to-end problems spanning the complete data science pipeline (data ingestion → cleaning → EDA → modeling → validation → deployment considerations)
- Incorporate big data processing scenarios requiring scalable computational approaches
- Verify solutions using Python with standard data science libraries and statistical methods
- Document problem statements clearly with realistic business contexts and provide verified correct answers
Freelance Editor (m/f/d)
- You create topic briefings, research and write technically sound (guide) texts in a sophisticated style, and edit the contributions of our freelance authors
- The topic areas include the hobby gardener target group in the garden and plant sector, as well as living and interior, design and décor, do-it-yourself, and cooking and nutrition are also part of the spectrum
- In close exchange with colleagues, readers and experts, you develop exciting topics and prepare them in a way that suits the audience
- Maintaining and expanding press contacts, as well as ordering photo material for garden, living and décor, are also among your tasks
- Optionally, you organize and conduct photo shoots, and attend press appointments and trade fairs
Developer for Consent Management Implementation (m/f/d)
To replace the web-based consent layers previously provided by third-party CMPs for our international brands, these layers need to be reimplemented in-house so we can operate and deploy them ourselves. This requires solid knowledge of TypeScript, Vue.js, and classic web technologies (HTML and CSS). The goal is to deliver executable code that meets all requirements and includes automated tests that prove correct functionality.
What the scope of work includes: The main focus is on developing decision-making elements for the implementation approach and carrying out measures along the resulting project path. This specifically includes the following deliverables:
- Implementation of code
- Implementation of executable tests that must pass for delivery, with test coverage >= 80%
- Creation of code documentation
- Creation of brand-specific cmp-config files
- Creation of a project (including asset management requirements) as a copy of the Consent Management Platform
- Removal of netID references
- Creation of brand-specific settings and files for custom purposes/providers
- Addition of new brand-specific CSS themes (variable values, logos, etc.)
- Inclusion of required official IAB GVL translations (ES, FR) in weekly synchronization with GVL
- Implementation of I18n and preparation of brand-specific data sources
- Implementation of PMC2.0 backend usage modules
- Implementation of playout logic
- Implementation of the layer initialization process (mode=default and mode=resurface)
- CDN upload and release process
- Project documentation
Project execution:
- The desired outcome should be written in TypeScript and Vue.js, built with Vite, tested with Vitest.
Chemist with Python Experience (m/w/d)
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as an AI Tutor in Chemistry, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
Project Manager for Magazines / Magazine Production (m/f/d)
- Responsibility for coordinating and controlling the entire production process of magazine publications
- Planning and monitoring issue structure, deadlines, advertisements and workflows
- Close collaboration with editorial, publishing management, marketing, technology, sales, printing house and service providers
- Quality assurance of layouts, texts and print approvals
- Cost calculation and organization of add-on products (e.g. inserts, posters, extensions)
- Active role in strategic projects, conferences and the launch of new formats
Data Engineer (m/f/d)
A company is looking for an experienced Data Engineer to carry out a migration from Snowflake to ClickHouse. The focus is on using Apache Spark for data processing and on managing and optimizing Kubernetes environments. The goal is to build and operate a powerful and scalable data platform.
- Executing the migration from Snowflake to ClickHouse
- Developing and optimizing data pipelines with Apache Spark
- Managing and optimizing Kubernetes clusters
- Ensuring the performance and scalability of the data platform
- Implementing solutions in Python
- Optional: Working with Snowplow for data analytics
Senior Project Manager Customer Interaction
A company is looking for support in a project to evaluate, implement and further develop quality surveys in digital channels. The goal of the project is to increase customer satisfaction in digital channels by evaluating, implementing and refining survey methods to enable consistent measurement of customer satisfaction across all channels. Improvement potentials should be identified and implemented.
The role includes advising, developing and implementing measures to collect and improve customer satisfaction in digital channels. Main tasks:
- Advising on survey methods for capturing customer experience and quality in digital channels, market standards, benchmarks and future direction.
- Developing a future model for quality in digital channels, relevant KPIs and survey methods as well as standard processes.
- Implementing the agreed measures including interface management and coordination with technology partners and social partners.
- Testing implemented measures to collect data and ensure required requirements are met.
- Consolidating and outlining existing and missing customer survey methods/quality KPIs in all managed digital channels.
- Advising on the preparation of decision papers and implementing the necessary measures.
- Identifying improvement potentials and developing a standard process for transparency and implementation.
New
Development of TM1 Planning Analytics and Interfaces (m/f/d)
A company is looking for support in the digital transformation of investment controlling. The role is part of a SCRUM team in the finance department. The focus is on developing and optimizing TM1 Planning Analytics as the core system, including building interfaces, data processing in the model, and providing data for reporting purposes.
Senior Web Developer (m/f/d)
- You develop modern, high-performance web frontends with React, TypeScript, HTML and CSS
- You implement responsive designs with a focus on accessibility and performance
- You plan and run unit and integration tests (for example with Playwright)
- Troubleshoot issues in development, test or live environments
Fullstack Engineer (m/f/d)
- Product and web development in a data-driven environment
- Shaping software architecture for new data products
- Collaborating in interdisciplinary teams (including data scientists and business developers)
Biologist with Python Experience (m/w/d)
GenAI models are improving very quickly, and one of our goals is to make them capable of addressing specialized questions and achieving complex reasoning skills. If you join the platform as an AI Tutor in Biology, you’ll have the opportunity to collaborate on these projects.
Although every project is unique, you might typically:
- Generate prompts that challenge AI.
- Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers.
- Correct the model’s responses based on your domain-specific knowledge.
Freelance Product Owner for Point of Sale App
- Vision & Strategy: You shape the roadmap for the POS system of the future and make sure we don't just meet our customers' needs, but anticipate them.
- Multi-Platform Excellence: You steer the development of our app landscape with a focus on Android. You understand the challenges of different form factors (from compact card terminals to large stationary tablets).
- Delivery Management: You lead your cross-functional development team with clarity and enthusiasm. You facilitate ceremonies, remove blockers, and celebrate releases.
- Requirements Engineering: You translate complex business logic into precise technical requirements and user stories that leave no questions for developers.
Adobe Experience Cloud Consultant (m/f/d)
The Digital Analytics department uses the Adobe Experience Cloud to deliver personalized user experiences. The goal is to increase conversion rates and improve customer experience through targeted personalization and testing. The technical implementation is handled independently by specialized consultants.
- A key part of the role involves maintaining the existing implementation within the Adobe Experience Platform. This includes monitoring data and troubleshooting source connectors.
- When preparing new features for use in the Adobe Experience Platform, requirements and data are first mapped to Adobe’s Experience Data Model. This involves creating entity-relationship diagrams (ERDs) and contextualizing relevant data.
- Based on that, the corresponding schemas are set up within the platform and datasets are prepared for further use.
- Other tasks include designing and setting up new activation channels and additional data sources.
- New, business-relevant use cases are also developed across the different phases of the customer journey to drive targeted business value.
- Creating segments and performing error analysis using targeted SQL queries.
- Advising on all newly introduced processing activities, ensuring compliance with applicable data protection regulations, necessary internal approval processes, and documenting both legal and technical specifics.
IT Project Manager ServiceNow (Senior)
- A company in the energy and energy services sector is looking for an experienced IT Project Manager for a ServiceNow project.
- The goal of the project is to lead and successfully implement an enterprise ServiceNow project focused on ITSM and Customer Service Management (CSM).
- The role involves planning, controlling and ensuring a smooth project process in close collaboration with internal and external stakeholders.
- Operational and strategic service management of the ServiceNow platform
- Process ownership for ITSM and CSM (B2B & B2C)
- Process design, governance and continuous optimization
- Management of external providers and vendors
- Monitoring, KPI analysis and deriving improvements
- Ensuring stable platform operations
Dentist for Training AI Models (m/f/d)
For an AI lab we are looking for German-speaking dentists to train an AI model (Large Language Model - LLM).
As a consultant, you may be invited to take part in online projects to train the model in your domain of expertise.
This flexible role accommodates both experts seeking part-time engagement (minimum a few hours/week) and those interested in full-time opportunities.
Although every project is unique, you might typically:
- Collaborate with the AI lab to provide domain-specific knowledge in dentistry.
- Participate in online training sessions to enhance the AI model's understanding.
- Review and validate AI-generated content for accuracy and relevance.
- Offer insights and feedback to improve the model's performance.
- Engage in flexible project-based work, adapting to unique project requirements.
Senior Factor 10 Developer (IPS / IPM) (m/f/d)
An insurance company in Nuremberg is looking for a Senior Factor 10 Developer with expertise in IPS and IPM. The project involves developing and optimizing software solutions in the insurance domain, focusing on high performance and reliability. The role requires solid knowledge of Factor 10 and its applications in the insurance sector.
Key responsibilities:
- Develop and optimize applications using Factor 10, especially in IPS and IPM.
- Collaborate with cross-functional teams to ensure seamless integration and functionality.
- Analyze and troubleshoot complex technical issues.
- Provide technical guidance and mentoring to junior developers.
- Ensure compliance with industry standards and best practices.
Frontend developer to HR platform with Angular experience
Reach out to us if you are interested in working with us on the project.
Sign up to get access to more exciting projects that match your skills and preferences!
Evaluation Scenario Writer (m/w/d)
Industry
Information Technology (IT)
Area
Quality Assurance (QA)
Project info
- Daily rate290 - 640€
- Language
- English(Advanced)
- English
- Remote100%
Description
We’re looking for someone who can design realistic and structured evaluation scenarios for LLM-based agents. You’ll create test cases that simulate human-performed tasks and define gold-standard behavior to compare agent actions against. You’ll work to ensure each scenario is clearly defined, well-scored, and easy to execute and reuse. You’ll need a sharp analytical mindset, attention to detail, and an interest in how AI agents make decisions.
Although every project is unique, you might typically:
- Designing structured test scenarios based on real-world tasks.
- Defining the golden path and acceptable agent behavior.
- Annotating task steps, expected outputs, and edge cases.
- Working with devs to test your scenarios and improve clarity.
- Reviewing agent outputs and adapting tests accordingly
Requirements
- Bachelor's and/or Master’s Degreein Computer Science, Software Engineering, Data Science / Data Analytics, Artificial Intelligence / Machine Learning, Computational Linguistics / Natural Language Processing (NLP), Information Systems or other related fields.
- Background in QA, software testing, data analysis, or NLP annotation.
- Good understanding of test design principles (e.g., reproducibility, coverage, edge cases).
- Strong written communication skills in English.
- Comfortable with structured formats like JSON/YAML for scenario description.
- Can define expected agent behaviors (gold paths) and scoring logic.
- Basic experience with Python and JS.
- Curious and open to working with AI-generated content, agent logs, and prompt-based behavior.
- You are ready to learn new methods, able to switch between tasks and topics quickly and sometimes work with challenging, complex guidelines.
- Our freelance role is fully remote so, you just need a laptop, internet connection, time available and enthusiasm to take on a challenge.
Nice to Have
- Experience in writing manual or automated test cases.
- Familiarity with LLM capabilities and typical failure modes.
- Understanding of scoring metrics (precision, recall, coverage, reward functions).