Associate Lead - Testing (QA + MLOps)
Read straight from Quantiphi's careers page, verified 13h ago.Seekless can tailor your résumé to it.
Seekless can write your résumé for this posting first.
Before you apply
LocationIN KA Bengaluru; IN MH Mumbai Eureka
Work modehybrid
Typefull-time
SeniorityLead
Experience10–12 yrs
Company size1,001–5,000 people
PostedMar 24, 2026 · 7mo ago
Verified live13h ago
At a glanceSummarised by Seekless from the posting.
8–12+ years in Quality Engineering
Strong test strategy, automation & governance experience
Experience leading QA transformation initiatives
Experience building frameworks from scratch
Deep understanding of ML lifecycle
Experience testing ML models (NLP preferred)
Hands-on experience validating LLM applications
Strong understanding of prompt engineering
Strong understanding of RAG architecture
Strong understanding of embeddings
Strong understanding of bias & explainability
Hands-on experience with Amazon SageMaker (training, deployment, monitoring)
Hands-on experience with Amazon Bedrock (LLM integration & evaluation)
Hands-on experience with S3-based data pipelines
Hands-on experience with AWS IAM (security validation)
Hands-on experience with CloudWatch monitoring
Hands-on experience with Lambda & API Gateway integrations
AWS CI/CD (CodePipeline / CodeBuild preferred)
Understanding of Infrastructure as Code (Terraform / CloudFormation)
Understanding of Observability in AI systems
Understanding of Cost monitoring for ML workloads
Python (mandatory)
Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch)
Experience with LLM frameworks (LangChain, etc.)
Experience with API & automation testing frameworks
Git-based workflows
Strong client-facing communication
Experience leading QA teams
Ability to create strategy decks & solution proposals
Strong stakeholder management
Skills
Amazon SageMaker
Amazon Bedrock
AWS Lambda
Amazon API Gateway
Amazon Kinesis
AWS Glue
Amazon S3
Amazon CloudWatch
SageMaker Pipelines
SageMaker Model Monitor
SageMaker Feature Store
AWS DevOps
Python
Scikit-learn
TensorFlow
PyTorch
LangChain
Terraform
CloudFormation
AWS IAM
CodePipeline
CodeBuild
Git
About the company
While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth.
If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!
Role: Lead/Associate Lead – QA + MLOps & Generative AI
Experience: 10+ years
Location: Mumbai/Bangalore (Hybrid)
Key Responsibilities:
AI/ML & GenAI Testing Strategy (AWS Ecosystem)
Define testing approaches for AI systems built on AWS services such as:
•
Amazon SageMaker
•
Amazon Bedrock
•
AWS Lambda
•
Amazon API Gateway
•
Amazon Kinesis
•
AWS Glue
•
Amazon S3
•
Amazon CloudWatch
Design validation frameworks covering:
•
Model accuracy & performance validation
•
Data drift & concept drift detection
•
Hallucination detection for LLMs
•
Prompt robustness testing
•
RAG validation (retrieval accuracy + grounding)
•
Bias & fairness validation
•
Safety & toxicity testing
MLOps Quality Engineering (AWS-Centric)
Validate the end-to-end ML lifecycle including:
•
Data ingestion & feature pipelines
•
Model training & hyperparameter tuning
•
Model versioning & registry
•
Deployment validation
•
Canary & blue/green release validation
Work with AWS-native services such as:
•
SageMaker Pipelines
•
SageMaker Model Monitor
•
SageMaker Feature Store
•
Bedrock model evaluation workflows
•
CloudWatch-based observability
Implement CI/CD quality gates for ML pipelines integrated with AWS DevOps tools.
GenAI & Agentic AI Testing
Define quality engineering approaches for:
•
LLM-based applications using Amazon Bedrock
•
Prompt engineering validation
•
Multi-agent orchestration testing
•
Chatbot & Voice bot conversational testing
•
Intent classification validation
•
Conversation drift & fallback validation
•
API contract validation for LLM integrations
Build reusable evaluation harnesses for:
•
BLEU / ROUGE scoring
•
Embedding similarity scoring
•
Response consistency
•
Safety scoring frameworks
Framework & Capability Development
•
Design reusable AI testing accelerators
•
Create AWS-aligned AI test automation frameworks (Python-first)
•
Develop synthetic data generation strategies
•
Establish AI quality scorecards
•
Build an internal AI QA Center of Excellence
Client Engagement & Leadership
•
Lead AI/ML quality strategy workshops
•
Perform AI risk & readiness assessments
•
Present quality architecture to CXOs
•
Drive QA transformation programs
•
Mentor QA teams on AWS-based AI testing
•
Own delivery for AI testing engagements end-to-end
Must have skills:
Testing Expertise
•
8–12+ years in Quality Engineering
•
Strong test strategy, automation & governance experience
•
Experience leading QA transformation initiatives
•
Experience building frameworks from scratch AI/ML & GenAI Expertise
•
Deep understanding of ML lifecycle
•
Experience testing ML models (NLP preferred)
•
Hands-on experience validating LLM applications
•
Strong understanding of:
•
Prompt engineering
•
RAG architecture
•
Embeddings
•
Bias & explainability AWS AI/ML Expertise
•
Hands-on experience with:
•
Amazon SageMaker (training, deployment, monitoring)
•
Amazon Bedrock (LLM integration & evaluation)
•
S3-based data pipelines
•
AWS IAM (security validation)
•
CloudWatch monitoring
•
Lambda & API Gateway integrations
•
AWS CI/CD (CodePipeline / CodeBuild preferred)
Understanding of:
•
Infrastructure as Code (Terraform / CloudFormation)
•
Observability in AI systems
•
Cost monitoring for ML workloads
Technical Skills
•
Python (mandatory)
•
Experience with ML libraries (Scikit-learn, TensorFlow, PyTorch)
•
Experience with LLM frameworks (LangChain, etc.)
•
API & automation testing frameworks
•
Git-based workflows
•
Leadership & Communication
•
Strong client-facing communication
•
Experience leading QA teams
•
Ability to create strategy decks & solution proposals
•
Strong stakeholder management
If you like wild growth and working with happy, enthusiastic over-achievers, you’ll enjoy your career with us*!*