What You’ll Do (Key Responsibilities)
1) Build the Quality Strategy and QA Operating System
• Perform a deep audit of the current QA setup across ARBI, Athena, frontend flows, backend services, APIs, data workflows, integrations, AI pipelines, and release processes.
• Define a company-wide QA strategy across short-term stabilization, mid-term automation, and long-term quality engineering maturity.
• Design a scalable test architecture using test pyramid principles, shift-left testing, smoke testing, regression testing, release gates, exploratory testing, and risk-based coverage.
• Define clear QA responsibilities between developers, QA, product, design, customer success, and release owners.
• Establish a practical quality operating rhythm: test plans, release checklists, defect triage, severity definitions, sign-off workflows, and quality metrics.
2) Own Product QA for ARBI and Athena
• Validate implementation against requirements, designs, copy, acceptance criteria, user stories, and customer-specific workflows.
• Test UI, UX, business logic, responsiveness, edge cases, error states, empty states, loading states, accessibility, and validation messages.
• Perform exploratory, smoke, regression, and release-candidate testing before launches.
• Validate recruiter workflows including candidate search, matching, scoring, resume parsing, outreach sequencing, scheduling, recruiter dashboards, candidate profiles, and ATS/CRM-style workflows.
• Validate Athena workflows including resume support, interview preparation, rubric-based feedback, candidate assistance, AI-generated recommendations, and user-facing guidance.
3) Build and Modernize Test Automation
• Take ownership of automated frontend, API, integration, and end-to-end test coverage using Cypress, Playwright, Pytest, Postman/Newman, or equivalent tools.
• Create reliable automated regression suites for critical ARBI and Athena workflows, including authentication, permissions, candidate pipelines, analytics, notifications, integrations, and admin experiences.
• Integrate tests deeply into CI/CD pipelines so failures are visible, actionable, and tied to release confidence.
• Improve test reliability, execution speed, data setup, fixture management, and maintainability.
• Introduce AI-assisted testing practices where useful, while maintaining clear human judgment and repeatable test evidence.
4) Validate Backend, API, Data, and Workflow Reliability
• Test robust REST APIs, Python/FastAPI services, backend business logic, asynchronous workflows, and data-processing pipelines.
• Validate PostgreSQL, Redis, OpenSearch, Celery, Temporal, containerized services, deployment pipelines, and AWS-hosted environments from a QA perspective.
• Create API and integration test coverage for imports, exports, webhooks, permissions, search, scoring, candidate data, customer-specific configuration, and workflow automation.
• Test with realistic and large-scale datasets to uncover performance, latency, search relevance, data integrity, and resilience issues.
• Establish baseline performance, load, and reliability testing using JMeter, k6, Locust, or similar tools.
5) QA AI, LLM, and Evaluation Workflows
• Validate AI-assisted recruiting workflows for accuracy, consistency, explainability, hallucination risk, bias risk, prompt adherence, rubric alignment, and human-in-the-loop behavior.
• Test AI scoring, candidate summaries, recommendations, interview feedback, resume analysis, and knowledge-retrieval experiences across normal, adversarial, and edge-case inputs.
• Create repeatable evaluation datasets and test harnesses to measure AI quality over time.
• Validate guardrails, fallback behavior, citations, confidence indicators, data boundaries, audit trails, and customer-specific configuration.
• Partner with product and engineering to define what “good” means for AI-generated outputs and how release readiness should be measured.
6) Own Release Readiness and Quality Visibility
• Create clear, structured, reproducible bug reports with screenshots/videos, environment details, severity, expected behavior, actual behavior, impact, and reproduction steps.
• Retest fixed issues, validate root-cause resolution, and prevent regressions.
• Communicate QA status clearly before release: passed, failed, blocked, passed with known issues, or requires founder/product decision.
• Build dashboards and reporting for test coverage, defect trends, regression health, release risk, performance baselines, and customer-impacting quality issues.
• Help developers produce testable, high-quality code by establishing testing standards, review practices, and shared quality expectations.