Lead Software Test Automation Engineer | Test Architect - Agentic AI
Salary$90k – $157k
LocationUSA - Coppell, TX; USA - Tampa, FL; USA - Minneapolis, MN; USA - Austin, TX; USA - Riverwoods, IL; USA - Kennesaw, GA; USA - Houston, TX, Allen Pkwy
Work modehybrid
Typefull-time
SeniorityLead
Experience8–12 yrs
Company size10,000+ people
First seenOct 7, 2026 · 4d ago
Verified live1d ago
At a glanceSummarised by Seekless from the posting.
Must have9
8–12+ years in Quality Engineering / Test Automation with architect-level ownership of frameworks and strategy
Proven experience designing scalable automation across API testing (contract + integration)
Proven experience designing scalable automation across UI automation with stable locator strategies (test IDs, accessibility roles)
Proven experience designing scalable automation across test data strategy and environment reliability
Strong programming skills in at least one modern language (e.g., Java, Python, TypeScript/JavaScript, C#)
Expertise with automation tools/frameworks: UI (Playwright / Cypress / Selenium)
Expertise with automation tools/frameworks: API (REST-assured / SuperTest / Postman/Newman / pytest)
Expertise with automation tools/frameworks: CI (GitHub Actions / Jenkins / Azure DevOps)
Demonstrated ability to reduce flakiness and improve execution speed through engineering solutions
Nice to have5
Experience testing LLM/Agentic AI features or other probabilistic systems, including evaluation harnesses, benchmark datasets, semantic scoring, and drift detection
Experience testing LLM/Agentic AI features or other probabilistic systems, including prompt/version regression strategies and structured evaluation reports
Familiarity with modern quality approaches: test pyramid, contract testing, consumer-driven contracts, risk-based testing, shift-left
Knowledge of security/performance testing integration (baseline understanding; hands-on is a plus)
Experience building reusable automation libraries for multiple teams/products
Skills
Java
Python
TypeScript
JavaScript
C#
Playwright
Cypress
Selenium
REST-assured
SuperTest
Postman
Newman
pytest
GitHub Actions
Jenkins
Azure DevOps
Benefits
Medical
Dental
Vision plans
401(k)
FSA
HSA
Commuter benefits
Tuition assistance plan
Vacation and sick time
Paid parental leave
We are seeking a Lead Test Automation Architect to define and lead an enterprise-grade test automation strategy across UI, API, data, and platform layers—while evolving our quality engineering capabilities to support Agentic AI / LLM-based systems (probabilistic outputs, confidence-based validation, hallucination detection, and model evaluation). This role will architect scalable automation frameworks, drive engineering excellence, and accelerate delivery through highly productive, maintainable automation programs.
Key Responsibilities
Automation Strategy & Architecture
•
Define and own the test automation architecture and multi-layer strategy (UI/API/contract/integration/e2e), aligned to product risk and release cadence.
•
Establish automation standards, patterns, and reference implementations (framework structure, page-object/screenplay usage, test data patterns, reporting).
•
Drive shift-left testing practices (contract testing, component testing, pre-merge validation) and reduce reliance on fragile end-to-end tests.
Agentic AI / LLM Test Model Alignment
•
Design and implement testing approaches for non-deterministic AI outputs, including:
•
confidence/threshold-based assertions
•
semantic similarity checks and rubric-based validation
•
goldens/benchmark datasets and evaluation harnesses
•
hallucination and groundedness testing strategies
•
regression detection across model/prompt/version changes
•
Define AI test coverage across accuracy, robustness, bias/safety (as applicable), performance, and reliability.
•
Partner with ML/AI, Product, and Engineering to establish acceptance criteria for AI capabilities and measurable quality gates.
High-Productivity Automation Programs
•
Build and scale high-throughput automation projects that are reliable, fast, and easy to extend.
•
Improve test execution time, reduce flaky tests, and optimize CI/CD pipelines (parallelization, test selection, retries with guardrails).
Demonstrated ability to reduce flakiness and improve execution speed through engineering solutions.
Preferred Qualifications (Strongly Desired)
•
Experience testing LLM/Agentic AI features or other probabilistic systems, including:
•
evaluation harnesses, benchmark datasets, semantic scoring, and drift detection
•
prompt/version regression strategies and structured evaluation reports
•
Familiarity with modern quality approaches: test pyramid, contract testing, consumer-driven contracts, risk-based testing, shift-left.
•
Knowledge of security/performance testing integration (baseline understanding; hands-on is a plus).
•
Experience building reusable automation libraries for multiple teams/products.
Our Interview Practices
To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process.
Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.
Compensation:
$89,600.00 - $157,000.00 USDThis role is eligible for Bonus.
Compensation range listed is based on primary location of the position. Actual base salary offer is influenced by a wide array of factors including but not limited to skills, experience and actual hiring location. Your recruiter can share more information about the specific offer for the job location during the hiring process.
Additional Information:
Wolters Kluwer offers a wide variety of competitive benefits and programs to help meet your needs and balance your work and personal life, including but not limited to: Medical, Dental, & Vision Plans, 401(k), FSA/HSA, Commuter Benefits, Tuition Assistance Plan, Vacation and Sick Time, and Paid Parental Leave. Full details of our benefits are available upon request.