Senior QA Engineer - AI Products (6 Months Contract)
Read straight from Arabic.AI's careers page, verified 15h ago.Seekless can tailor your résumé to it.
Seekless can write your résumé for this posting first.
Before you apply
LocationDoha, Doha Municipality, Qatar
Typecontract
SenioritySenior
Experience7+ yrs
DepartmentTechnology, AI, Products & PMO
Company size51–200 people
PostedOct 27, 2026 · 1w ago
Verified live15h ago
At a glanceSummarised by Seekless from the posting.
7+ years of experience in QA / software testing, with strong test automation background
Proven experience testing AI/ML or LLM-based products (chatbots, RAG, agents, etc.)
Hands-on scripting experience in Python and/or TypeScript/JavaScript
Hands-on with at least one UI automation framework (Playwright, Cypress, Selenium) and a test framework such as Pytest
API testing with Postman, REST Assured, or similar
Experience with LLM evaluation tools
CI/CD and version control: GitHub Actions, GitLab CI or Jenkins; Git
Test management and tracking: Jira, TestRail, or Zephyr
Excellent written English and clear bug reporting
Performance/load testing (k6, JMeter, Locust)
Understanding of prompt engineering, embeddings, and vector databases
Experience with AI safety, red-teaming, or content policy testing
Cloud experience (AWS, GCP, or Azure) and Docker
Skills
Python
TypeScript
JavaScript
Playwright
Cypress
Selenium
Pytest
Postman
REST Assured
GitHub Actions
GitLab CI
Jenkins
Git
Jira
TestRail
Zephyr
k6
JMeter
Locust
Docker
AWS
GCP
Azure
About the role
We are looking for a Senior QA Engineer to own quality across our AI products. You will test LLM-powered features end to end, build automated test and evaluation suites, and help ensure every release is accurate, safe, and reliable.
Responsibilities
Responsibilities
• Define test strategies and plans for AI features across web, API, and backend.
• Evaluate model outputs for accuracy, consistency, hallucinations, bias, and safety.
• Build evaluation datasets, including edge cases and adversarial / red-team prompts.
• Develop and maintain automated UI, API, and regression tests integrated into CI/CD.
• Run regression testing on model, prompt, and configuration changes.
• Test performance, latency, error handling, and fallback behavior.
• Report defects clearly and provide release-readiness sign-off.
• Test non-deterministic outputs using scoring rubrics, LLM-as-judge, human evaluation, and defined pass/fail thresholds.
• Perform AI security testing, including prompt injection, jailbreaks, and PII / data leakage across chatbots, RAG, and agents.
• Work with ML engineers and product managers to define quality metrics and acceptance criteria.
• Where applicable, monitor production quality and model drift after release.
Requirements
7+ years of experience in QA / software testing, with strong test automation background.
• Proven experience testing AI/ML or LLM-based products (chatbots, RAG, agents, etc.).
• Hands-on scripting experience in Python and/or TypeScript/JavaScript.
• Hands-on with at least one UI automation framework (Playwright, Cypress, Selenium) and a test framework such as Pytest.
• API testing with Postman, REST Assured, or similar.
• Experience with LLM evaluation tools.
• CI/CD and version control: GitHub Actions, GitLab CI or Jenkins; Git.
• Test management and tracking: Jira, TestRail, or Zephyr.
• Excellent written English and clear bug reporting.
Nice to have
Nice to Have
• Performance/load testing (k6, JMeter, Locust).
• Understanding of prompt engineering, embeddings, and vector databases.
• Experience with AI safety, red-teaming, or content policy testing.
• Cloud experience (AWS, GCP, or Azure) and Docker.