AI/ML Test Engineer

• Execute system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing.
• Perform integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows.
• Support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment.
• Develop, maintain, and refine data annotation workflows supporting operational evaluation requirements.
• Validate operational datasets and support testing activities across multiple models, datasets, and mission scenarios.
• Conduct integration testing of AI/ML models within operationally representative environments and workflows.
• Adapt evaluation procedures and workflows to address evolving GEOINT data conditions and operational mission requirements.
• Support high-throughput and parallel model evaluation activities to maintain alignment with mission timelines and operational priorities.
• Identify and mitigate data drift, performance degradation, and workflow issues impacting evaluation reliability.
• Collaborate with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders to support scalable T&E execution.
• Contribute to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.

• 4+ years of experience supporting AI/ML testing, system integration, operational testing, or evaluation workflows.
• Familiarity with automated testing pipelines, operational data validation, and model performance assessment concepts.
• Ability to support multiple evaluation activities across dynamic operational environments.
• Experience troubleshooting technical workflow or data-related issues in support of operational objectives.
• Strong analytical, organizational, and communication skills.

• Bachelor's degree in IT or STEM related field, or additional relevant experience in lieu of a degree.

• Experience supporting DoD, NGA, or Intelligence Community programs.
• Familiarity with AI/ML model evaluation, GEOINT workflows, and operational testing environments.
• Experience working in Agile, DevSecOps, or cloud-based operational environments.
• Familiarity with annotation workflows, data capture pipelines, drift detection, and automated evaluation tools.
• Experience supporting scalable AI/ML evaluation or mission-focused T&E activities.

• Active TS/SCI clearance and the ability to pass a CI polygraph within 30 days.

• 
  
  
  
  Interesting Work:
  
  
  
  Our co-workers support some of the most important and critical programs to our national defense and security.
  
  
• 
  
  
  
  Values:
  
  
  
  Our first core value is that employees come first. We challenge our co-workers to provide the highest level of support and service, and reward them with some of the best benefits in the industry.
  
  
• 
  
  
  
  100% Employee Owned:
  
  
  
  We have a stake in each other's success, and the success of our customers. It's also nice to know what's going on across the company; we have company wide town-hall meetings three times a year.
  
  
• 
  
  
  
  Great Benefits - Most Full-Time Staff Are Eligible for:
  
  
  




• 
  Starting PTO accrual of 20 days PTO/year + 10 holidays/year
  
• 
  Flexible schedules
  
• 
  6% 401k match with immediate vesting up to $9k annually
  
• 
  Semi-annual bonus eligibility (July and December)
  
• 
  Company funded Employee Stock Ownership Plan (ESOP) - a separate qualified retirement account
  
• 
  Up to $10,000 in annual educational reimbursement
  
• 
  Other company funded benefits, like life and disability insurance
  
• 
  Optional zero deductible Blue Cross/Blue Shield health insurance plan
  




• 
  
  
  
  Track Record of Success:
  
  
  
  We have grown every year since our founding in 1993.
  
  

Modern Technology Solutions, Inc. (MTSI) is a 100% employee-owned engineering services and solutions company that provides high-demand technical expertise in Digital Transformation, Modeling and Simulation, Rapid Capability Development, Test and Evaluation, Artificial Intelligence, Autonomy, Cybersecurity and Mission Assurance

MTSI delivers capabilities to solve problems of global importance. Founded in 1993, MTSI today has employees at over 20 offices and field sites worldwide.

For more information about MTSI, please visit www.mtsi-va.com