Sr. Staff AI Engineer - On-Prem AI Infrastructure & Agentic Systems
Salary$140k – $165k
LocationSan Jose
Typefull-time
SeniorityLead
Experience2+ yrs
DepartmentSystem Engineering
Company size10,000+ people
First seenOct 4, 2026 · 1w ago
Verified live1d ago
At a glanceSummarised by Seekless from the posting.
Must have8
2+ years of experience in AI/ML engineering, with hands-on deployment of AI systems on-prem or private cloud
Proven experience building agentic AI systems — including state management, tool integration, and multi-step reasoning
Strong working knowledge of RAG architectures — chunking, retrieval, re-ranking, evaluation metrics
Experience with model fine-tuning (LoRA, QLoRA, full fine-tuning) and embedding models for retrieval
Familiarity with Model Control Protocols (MCP) or similar governance frameworks (model versioning, access control, audit trails)
Proficiency in Python, Linux, Docker/Kubernetes, and vector databases (e.g., Milvus, Qdrant, Pinecone)
Experience with AI serving frameworks (vLLM, TGI, Triton, Ollama, etc.)
Bachelor of Science in CS, EE, ME, or other applicable Engineering field
Nice to have6
Experience deploying AI in enterprise storage or hardware-adjacent environments
Background in systems engineering or QA automation — bonus if you've used AI to automate testing or validation
Familiarity with embedded AI or edge inference (ONNX, TensorRT, GGUF, etc.)
Experience with AI agent frameworks (LangGraph, AutoGen, BabyAGI, etc.)
Knowledge of AI observability tools (LangSmith, Weights & Biases, Prometheus/Grafana for AI)
Knowledge of storage area/NVMe
Skills
Python
Linux
Docker
Kubernetes
Helm
Milvus
Qdrant
FAISS
Pinecone
vLLM
TGI
Triton
Ollama
LoRA
QLoRA
LangGraph
AutoGen
ONNX
TensorRT
GGUF
LangSmith
Weights & Biases
Prometheus
Grafana
Benefits
Health insurance
Dental
Vision
Life insurance
401(k)
Cafeteria
Onsite gym
About the Company:
At SK Hynix Memory Solution, we’re at the forefront of semiconductor innovation, developing advanced memory solutions that power everything from smartphones to data centers. As a global leader in DRAM and NAND flash technologies, we drive the evolution of advancing mobile technology, empowering cloud computing, and pioneering future technologies. Our cutting-edge memory technologies are essential in today’s most advanced electronic devices and IT infrastructure, enabling enhanced performance and user experiences across the digital landscape.
We’re looking for innovative minds to join our mission of shaping the future of technology. At SK Hynix Memory, you’ll be part of a team that’s pioneering breakthrough memory solutions while maintaining a strong commitment to sustainability. We’re not just adapting to technological change – we’re driving it, with significant investments in artificial intelligence, machine learning, and eco-friendly solutions and operational practices. As we continue to expand our market presence and push the boundaries of what’s possible in semiconductor technology, we invite you to be part of our journey to creating the next generation of memory solutions that will define the future of computing.
Why Join Us?
•
Build foundational AI infrastructure that powers next-gen enterprise systems.
•
Work on cutting-edge agentic AI — not just chatbots, but autonomous systems that reason, plan, and act.
•
Opportunity to influence AI strategy, deployment, and governance in a high-impact environment.
About the Role:
We are seeking a hands-on AI Engineer to design, deploy, and maintain on-prem AI infrastructure and build agentic AI systems that drive real-world automation. You’ll be responsible for setting up scalable AI environments, implementing RAG pipelines, fine-tuning embedded models, and architecting AI agents that operate autonomously in enterprise settings. This role sits at the intersection of AI systems engineering and applied ML — you’ll bridge infrastructure, model deployment, and agent logic.
Responsibilities:
•
Design and deploy on-prem AI infrastructure — including GPU clusters, model serving (e.g., vLLM, TGI, Triton), vector DBs (e.g., Milvus, Qdrant, FAISS), and orchestration (Kubernetes, Helm, Docker).
•
Build and optimize RAG pipelines — including document chunking, retrieval strategies (hybrid, re-ranking), and evaluation of retrieval accuracy and latency.
•
Develop agentic AI systems — design stateful agents with memory, tool use, and planning capabilities (e.g., using LangGraph, AutoGen, or custom frameworks).
•
Fine-tune and deploy embedded models — work with LoRA, QLoRA, or full fine-tuning for domain-specific tasks; optimize for edge/on-device inference.
•
Implement Model Control Protocols (MCP) — ensure model governance, versioning, access control, and monitoring for production AI systems.
•
Collaborate with product and engineering teams to integrate AI capabilities into enterprise workflows — especially in storage, QA, or systems engineering contexts.
•
Automate and monitor AI pipelines — build CI/CD for model deployment, logging, and performance tracking.
Minimum Qualifications:
•
2+ years of experience in AI/ML engineering, with hands-on deployment of AI systems on-prem or private cloud.
•
Proven experience building agentic AI systems — including state management, tool integration, and multi-step reasoning.
•
Strong working knowledge of RAG architectures — chunking, retrieval, re-ranking, evaluation metrics.
•
Experience with model fine-tuning (LoRA, QLoRA, full fine-tuning) and embedding models for retrieval.
•
Familiarity with Model Control Protocols (MCP) or similar governance frameworks (model versioning, access control, audit trails).
•
Proficiency in Python, Linux, Docker/Kubernetes, and vector databases (e.g., Milvus, Qdrant, Pinecone).
•
Experience with AI serving frameworks (vLLM, TGI, Triton, Ollama, etc.).
Preferred Qualifications:
•
Experience deploying AI in enterprise storage or hardware-adjacent environments.
•
Background in systems engineering or QA automation — bonus if you’ve used AI to automate testing or validation.
•
Familiarity with embedded AI or edge inference (ONNX, TensorRT, GGUF, etc.).
•
Experience with AI agent frameworks (LangGraph, AutoGen, BabyAGI, etc.).
•
Knowledge of AI observability tools (LangSmith, Weights & Biases, Prometheus/Grafana for AI).
•
As a Storage company, knowledge of storage area/NVMe is a PLUS.
Education Requirement:
•
Bachelor of Science in CS, EE, ME, or other applicable Engineering field.
COMPENSATION: $140,000/yr - $165,000/yr
REGARDING COMPENSATION:
SK hynix memory solutions America Inc. offers you the opportunity to apply your skills to exciting projects while working with innovative teams. Our compensation package is complimented by a generous benefits package including medical, dental, vision, life insurance and a company 401(k) match, as well as cafeteria, onsite gym and much more. If you are motivated by technical challenges, we offer a collaborative work environment that encourages career growth.
Legal
The salary offered to a selected candidate will be tailored based on several factors, including the location, job grade, relevant knowledge, skills, and experience. We also take into account the internal equity among our current team members to ensure fairness and competitiveness
More roles at SK hynix memory solutions America Inc.