Senior Software Engineer - AI Test Environments
Posted date [2026-08-19]
(ID: 49849)
 |
YO IT Consulting specializes in recruiting remote professionals for AI training and data annotation projects with global companies. Based in Abu Dhabi Emirate, United Arab Emirates, the company connects qualified talent worldwide with flexible remote opportunities that contribute to the development and improvement of modern artificial intelligence systems. |
Job Description and Requirements
Senior Software Engineer - AI Test EnvironmentsJob Snapshot
Role: Senior Software Engineer - AI Test Environments
Location: Dubai, United Arab Emirates
Industry: Computer Software
Function: Software Engineering / Development
Experience: Senior-level software engineering experience
Job Type: Contractor
Position Overview
The Senior Software Engineer - AI Test Environments opportunity in Dubai, United Arab Emirates is a remote Computer Software contract offered through YO IT Consulting. Working approximately 15 hours per week, the engineer will create reinforcement learning environments that test how effectively AI agents solve realistic software problems using Model Context Protocol tools.
Job Details
Country: United Arab Emirates
City: Dubai
Industry: Computer Software
Function: Software Engineering / Development
Salary: 8000-18000
Estimated salary range based on similar jobs in Dubai; please confirm the final offer with the employer.
Gender: Any
Candidate Nationality: Any
Job Type: Contractor
Role Context
This project combines advanced software engineering with AI capability evaluation. The engineer will
build controlled scenarios involving bug resolution, feature development, refactoring, and performance optimization while requiring AI agents to discover and use information from real Model Context Protocol servers.
Every environment must be reproducible and supported by deterministic verification and a reliable golden-reference solution. These controls allow project teams to measure both software-engineering competence and correct tool usage.
Compensation is based on accepted task output rather than hours worked. Minimum weekly submission requirements apply, and the time needed for each assignment will depend on its complexity and the contributor\'s workflow.
Key Responsibilities
* Design reinforcement learning environments around realistic software-engineering challenges.
* Create tasks that require AI agents to interact with Model Context Protocol tools and servers.
* Develop scenarios involving defect correction, new features, code modernization, and performance improvement.
* Define clear task requirements, constraints, repository state, and expected outcomes.
* Build reproducible environments that behave consistently across repeated evaluation runs.
* Establish deterministic checks that verify whether an AI-generated solution is correct.
* Create golden-reference implementations demonstrating an approved solution path.
* Ensure evaluation criteria measure both engineering quality and effective tool usage.
* Develop test suites covering functional behaviour, failure conditions, and important edge cases.
* Confirm that tasks cannot be completed correctly through unsupported shortcuts or ambiguous assumptions.
* Introduce realistic information-discovery requirements using available Model Context Protocol resources.
* Debug environment failures and separate infrastructure issues from agent-performance problems.
* Review generated code for correctness, maintainability, scalability, and runtime efficiency.
* Refactor task assets and verification logic when evaluation behaviour is inconsistent.
* Optimize evaluation environments for reliability and repeatable execution.
* Document setup steps, tool dependencies, intended reasoning paths, and scoring logic.
* Participate in technical reviews and respond to calibration feedback.
* Collaborate remotely with engineering, research, and AI evaluation teams.
* Meet agreed task-quality standards and minimum weekly submission expectations.
* Maintain careful version control across environments, tests, and reference solutions.
Ideal Profile
Candidates should have senior-level proficiency in at least one of these languages: C++, Python, Java, Go, TypeScript, or Rust. Strong understanding of algorithms, data structures, debugging, performance tuning, feature development, and codebase refactoring is required.
The role suits an engineer who can create maintainable solutions, work confidently with large or distributed repositories, and explain technical decisions precisely. Experience participating in detailed code reviews and improving software-engineering practices is preferred.
Previous AI or machine learning experience is helpful but not mandatory. Candidates should be comfortable working independently in a remote environment and be available to begin assignments shortly after onboarding. The selection process may include screening questions, an AI interview, a technical assessment, and hiring-manager review.
Skills Set
* Reinforcement learning environments
* Model Context Protocol
* MCP tool integration
* AI agent evaluation
* Software task design
* Deterministic verification
* Golden-reference solutions
* Reproducible test environments
* C++
* Python
* Java
* Go
* TypeScript
* Rust
* Algorithms and data structures
* Complex debugging
* Feature implementation
* Codebase refactoring
* Performance optimization
* Scalability engineering
* Automated testing
* Distributed codebases
* Technical documentation
* Code review
* Git version control
Why Join Us
This contract offers experienced engineers a direct role in determining how advanced AI agents are tested on genuine software-development work. The flexible part-time structure allows contributors to apply deep engineering judgement while gaining practical exposure to MCP-based tools, reinforcement learning evaluation, and emerging agentic development systems.
About the Company
YO IT Consulting connects senior technology professionals with specialized software and AI projects. This engagement focuses on building rigorous engineering environments that measure how accurately and reliably AI agents use tools to solve complex coding problems.
Every environment must be reproducible and supported by deterministic verification and a reliable golden-reference solution. These controls allow project teams to measure both software-engineering competence and correct tool usage.
Compensation is based on accepted task output rather than hours worked. Minimum weekly submission requirements apply, and the time needed for each assignment will depend on its complexity and the contributor\'s workflow.
Key Responsibilities
* Design reinforcement learning environments around realistic software-engineering challenges.
* Create tasks that require AI agents to interact with Model Context Protocol tools and servers.
* Develop scenarios involving defect correction, new features, code modernization, and performance improvement.
* Define clear task requirements, constraints, repository state, and expected outcomes.
* Build reproducible environments that behave consistently across repeated evaluation runs.
* Establish deterministic checks that verify whether an AI-generated solution is correct.
* Create golden-reference implementations demonstrating an approved solution path.
* Ensure evaluation criteria measure both engineering quality and effective tool usage.
* Develop test suites covering functional behaviour, failure conditions, and important edge cases.
* Confirm that tasks cannot be completed correctly through unsupported shortcuts or ambiguous assumptions.
* Introduce realistic information-discovery requirements using available Model Context Protocol resources.
* Debug environment failures and separate infrastructure issues from agent-performance problems.
* Review generated code for correctness, maintainability, scalability, and runtime efficiency.
* Refactor task assets and verification logic when evaluation behaviour is inconsistent.
* Optimize evaluation environments for reliability and repeatable execution.
* Document setup steps, tool dependencies, intended reasoning paths, and scoring logic.
* Participate in technical reviews and respond to calibration feedback.
* Collaborate remotely with engineering, research, and AI evaluation teams.
* Meet agreed task-quality standards and minimum weekly submission expectations.
* Maintain careful version control across environments, tests, and reference solutions.
Ideal Profile
Candidates should have senior-level proficiency in at least one of these languages: C++, Python, Java, Go, TypeScript, or Rust. Strong understanding of algorithms, data structures, debugging, performance tuning, feature development, and codebase refactoring is required.
The role suits an engineer who can create maintainable solutions, work confidently with large or distributed repositories, and explain technical decisions precisely. Experience participating in detailed code reviews and improving software-engineering practices is preferred.
Previous AI or machine learning experience is helpful but not mandatory. Candidates should be comfortable working independently in a remote environment and be available to begin assignments shortly after onboarding. The selection process may include screening questions, an AI interview, a technical assessment, and hiring-manager review.
Skills Set
* Reinforcement learning environments
* Model Context Protocol
* MCP tool integration
* AI agent evaluation
* Software task design
* Deterministic verification
* Golden-reference solutions
* Reproducible test environments
* C++
* Python
* Java
* Go
* TypeScript
* Rust
* Algorithms and data structures
* Complex debugging
* Feature implementation
* Codebase refactoring
* Performance optimization
* Scalability engineering
* Automated testing
* Distributed codebases
* Technical documentation
* Code review
* Git version control
Why Join Us
This contract offers experienced engineers a direct role in determining how advanced AI agents are tested on genuine software-development work. The flexible part-time structure allows contributors to apply deep engineering judgement while gaining practical exposure to MCP-based tools, reinforcement learning evaluation, and emerging agentic development systems.
About the Company
YO IT Consulting connects senior technology professionals with specialized software and AI projects. This engagement focuses on building rigorous engineering environments that measure how accurately and reliably AI agents use tools to solve complex coding problems.
