Senior AI QA Engineer for Sora Union
Read straight from Sora Union's careers page, verified 1d ago.Seekless can tailor your résumé to it.
Seekless can write your résumé for this posting first.
Before you apply
LocationSora Union, Global
Work moderemote
SenioritySenior
Experience7+ yrs
DepartmentEngineering
Company size8+ people
PostedFeb 14, 2025 · 1y ago
Verified live1d ago
May no longer be actively hiring
Posted by the company on Feb 14, 202575% of this company's open roles have been listed for 90+ days
At a glanceSummarised by Seekless from the posting.
A degree in computer science or mathematics
Minimum of 7 years of experience in a professional QA role
Hands-on experience with AI/LLM testing, including evaluating AI-generated responses, validating expected behavior, and identifying inconsistencies or failure scenarios
Practical experience using AI tools such as Claude and ChatGPT to support testing, test generation, debugging, or QA workflow automation
Experience with Playwright and MCP-based automation, including using Playwright MCP or similar approaches to automate and validate application workflows
Experience building or maintaining AI-powered testing agents, test harnesses, or automated evaluation workflows
Strong understanding of manual, exploratory, functional, smoke, sanity, regression, integration, and end-to-end testing
Experience testing web applications and backend APIs, including validating data flows and system integrations
Familiarity with test automation frameworks such as Playwright, Selenium, or Cypress
Experience with API testing tools such as Postman and a working knowledge of REST APIs and, ideally, GraphQL
Working knowledge of SQL and databases, including the ability to write and execute queries for data verification
Familiarity with defect tracking and test management tools such as Jira, TestRail, Azure DevOps, or GitHub
Understanding of Agile/Scrum methodologies and experience working in cross-functional software development teams
Strong analytical and problem-solving skills, with exceptional attention to detail and the ability to investigate complex issues
Ability to work independently, manage priorities, and adapt to evolving requirements and emerging technologies
Excellent written and verbal communication skills, with the ability to explain technical findings clearly to both technical and non-technical stakeholders
English - Upper-intermediate or higher
Skills
Claude
ChatGPT
Playwright MCP
Playwright
Selenium
Cypress
Postman
REST APIs
GraphQL
SQL
Jira
TestRail
Azure DevOps
GitHub
AI/LLM testing
Talent is everywhere, opportunity is not. Sora Union changes that.
Sora Union is a professional services company specializing in brand design, engineering services. The Sora Union team is globally-distributed and made up of diverse and talented professionals impacted by or at risk of displacement due to conflict or climate change.
About the role
We’re looking for an experienced Senior AI QA Engineer to join us to help expand our team of diverse and experienced professionals.
As a AI QA Engineer working on projects in partnership with our clients, you’ll work with products and companies both big and small, primarily responsible for ensuring they deliver the highest quality experience possible for their users. You’ll work with software developers, product managers, designers, and customer service teams to increase confidence in our clients’ releases, including testing applications, investigating issues, and reporting on results.
You’ll be successful in this role if you enjoy working towards becoming one of the most knowledgeable individuals about a product and how it gets to consumers. You should be an effective communicator, comfortable exchanging information in a clear and concise way with both technical and non-technical stakeholders.
You’ll be responsible for:
•
AI and LLM Testing: Design and execute tests to evaluate LLM-generated responses for accuracy, relevance, consistency, completeness, instruction adherence, and potential failure cases.
•
AI Agent Development: Build, maintain, and improve cost-effective AI-powered testing agents and test harnesses to support software development and delivery.
•
MCP Testing: Test Model Context Protocol (MCP) tools and integrations, validating tool execution, input/output handling, expected behavior, and error scenarios.
•
AI-Driven Automation: Use Claude, ChatGPT, Playwright MCP, and related tools to streamline test creation, execution, analysis, and reporting.
•
Test Strategy and Execution: Develop and maintain comprehensive test plans, test cases, regression suites, and manual testing checklists.
•
Automation and Regression Testing: Design, execute, and maintain automated tests for web applications, APIs, and end-to-end workflows.
•
Functional and Exploratory Testing: Investigate product behavior, identify edge cases, and uncover functional, usability, integration, and data-related issues.
•
API and Data Validation: Test backend APIs, verify data integrity, work with SQL queries, and validate system integrations and data flows.
•
Defect Management: Reproduce, document, triage, prioritize, and track defects, providing clear evidence and actionable information to development teams.
•
Cross-Functional Collaboration: Partner with engineering, product, and other teams to clarify requirements, assess risks, and improve release confidence.
•
Continuous Improvement: Identify opportunities to improve QA processes, expand test coverage, reduce repetitive manual work, and introduce practical AI-driven testing solutions.
•
Product Expertise: Develop a deep understanding of the product, its users, and its underlying workflows to ensure testing reflects real-world usage
Ideally, you’ll have:
•
A degree in computer science or mathematics, coupled with a minimum of 7 years of experience in a professional QA role.
•
Hands-on experience with AI/LLM testing, including evaluating AI-generated responses, validating expected behavior, and identifying inconsistencies or failure scenarios.
•
Practical experience using AI tools such as Claude and ChatGPT to support testing, test generation, debugging, or QA workflow automation.
•
Experience with Playwright and MCP-based automation, including using Playwright MCP or similar approaches to automate and validate application workflows.
•
Experience building or maintaining AI-powered testing agents, test harnesses, or automated evaluation workflows.
•
Strong understanding of manual, exploratory, functional, smoke, sanity, regression, integration, and end-to-end testing.
•
Experience testing web applications and backend APIs, including validating data flows and system integrations.
•
Familiarity with test automation frameworks such as Playwright, Selenium, or Cypress.
•
Experience with API testing tools such as Postman and a working knowledge of REST APIs and, ideally, GraphQL.
•
Working knowledge of SQL and databases, including the ability to write and execute queries for data verification.
•
Familiarity with defect tracking and test management tools such as Jira, TestRail, Azure DevOps, or GitHub.
•
Understanding of Agile/Scrum methodologies and experience working in cross-functional software development teams.
•
Strong analytical and problem-solving skills, with exceptional attention to detail and the ability to investigate complex issues.
•
Ability to work independently, manage priorities, and adapt to evolving requirements and emerging technologies.
•
Excellent written and verbal communication skills, with the ability to explain technical findings clearly to both technical and non-technical stakeholders.
•
English - Upper-intermediate or higher
Legal
Sora Union is committed to creating and fostering a diverse team. We encourage people from underrepresented backgrounds and all walks of life to apply. We are committed to providing reasonable accommodations to all applicants throughout the application process.