Senior AI Engineer - LLM Adoption Department (LLMAD)

LocationTokyo, Japan

Work modeon-site

Typefull-time

SenioritySenior

Experience6+ yrs

DepartmentAI & Data Division

Company size10,000+ people

PostedAug 21, 2026 · 1mo ago

Verified live10h ago

At a glanceSummarised by Seekless from the posting.

Over 6 years of professional software engineering experience, with recent hands-on focus on building LLM applications

Proven agentic engineering experience: built, shipped, evaluated, and debugged agents in production

Prompt and context engineering as a discipline: structured outputs, systematic iteration against evals, and versioning

Practical model knowledge: strengths and weaknesses of open-weight and commercial models, trade-offs of fine-tuning vs prompting vs retrieval, cost and latency trade-offs

Solid production fundamentals: APIs, backend services, testing, observability, and experience with self-hosted or cloud inference (e.g., vLLM)

Able to work embedded in someone else's engineering team, communicate directly with business stakeholders, and operate with ambiguity

Writes things down - docs, examples, and recipes other engineers can actually use

Japanese language ability - a significant plus

Experience migrating production applications from third-party LLM APIs to self-hosted or open-weight models

Retrieval-augmented generation at production scale

Exposure to fine-tuning or post-training, enough to know when to reach for it

Experience delivering with distributed teams in Japan or Asia

Skills

LLM

Python

vLLM

RAG

APIs

backend services

testing

observability

fine-tuning

prompt engineering

Job Description:

Business Overview

Rakuten Group, Inc. is a global leader in internet services that empower individuals, communities, businesses, and society. Founded in Tokyo in 1997 as an online marketplace, Rakuten has expanded to offer services in e-commerce, fintech, digital content, and communications to approximately 1.7 billion members around the world. The Rakuten Group has nearly 32,000 employees and operations in 30 countries and regions. For more information visit https://global.rakuten.com/corp/

Department Overview

The AI & Data Division (AIDD) creates powerful, customer-focused search, recommendation, data science, advertising, marketing, price, and inventory optimization solutions to a variety of businesses in the Rakuten group. Our goal is to drive innovation by developing new products and capabilities that deliver significant impact over longer timeframes using AI.

About the role

The LLMAD team drives adoption of Rakuten AI and open-source LLMs hosted within our own environment. We’re hiring AI Engineers who can be embedded directly inside business units to drive real adoption of in-house large language models in place of third-party alternatives.

Position:

Position Details

As a Senior AI Engineer you will own a business unit engagement end to end. You sit inside their engineering team and build with them: designing the agent, selecting and evaluating the model, writing the prompts and tool-use logic, and getting it live in production on our own infrastructure.

You are expected to operate independently - taking a vague business ask, turning it into a concrete and evaluable LLM task, and being accountable for it shipping. What you build also becomes reusable: the cookbooks, recipes, and reference implementations the rest of the team and other business units start from.

Responsibilities

・Own a business unit engagement end to end - embed in their engineering team, design the agent (task decomposition, tool and function calling, multi-step orchestration, state, retrieval, guardrails), and write production code in their codebase.

・Take agents from prototype to production and own what that requires: latency, cost, failure modes, and monitoring - including the judgment to say early when an agent is the wrong answer and a prompt, a fine-tune, or plain software is not.

・Select the right in-house or open-source model for each task and prove it with task-level evaluation rather than benchmark scores, building the eval sets and harnesses together with the business unit.

・Run migrations from third-party APIs to our in-house models - parity testing, prompt porting, staged cutover - and feed the gaps you find back to our model and platform teams.

・Own prompt and context engineering as a discipline: structured output, tool schemas, retrieval strategy, and caching, versioned and measured against evals; debug quality problems down to whether the fault is the prompt, the retrieval, the model, or the data.

・Turn the work into cookbooks, recipes, and reference implementations other business units can lift directly, and raise the bar around you through code review, mentorship, and hands-on enablement for business unit engineers.

This role sits close to the product rather than close to the model:

・You will not be training or fine-tuning foundation models - that sits with our model research and training teams.

・You will not be building the LLM platform itself - that sits with our platform engineering teams.

・You will be embedded in a business unit, writing agent and integration code inside their product, and accountable for it shipping on our models.

Mandatory Qualifications:

・Over 6 years of professional software engineering experience, with recent hands-on focus on building LLM applications. You are shipping this work today.

・Proven agentic engineering experience: you have built, shipped, evaluated, and debugged agents in production - tool and function calling, multi-step orchestration, and the failure modes that come with them.

・Prompt and context engineering as a discipline: structured outputs, systematic iteration against evals, and versioning.

・Practical model knowledge: the strengths and weaknesses of open-weight and commercial models, when to fine-tune versus prompt versus retrieve, and the cost and latency trade-offs of each.

・Solid production fundamentals: APIs, backend services, testing, observability, and experience with self-hosted or cloud inference (e.g., vLLM).

・Able to work embedded in someone else’s engineering team, communicate directly with business stakeholders, and operate with the ambiguity of a function that is still being defined.

・Writes things down - docs, examples, and recipes other engineers can actually use.

Desired Qualifications:

・Japanese language ability - a significant plus. Our business unit teams work in Japanese day to day, and being able to work in the language directly makes the embedded model far more effective.

・Experience migrating production applications from third-party LLM APIs to self-hosted or open-weight models.

・Retrieval-augmented generation at production scale.

・Exposure to fine-tuning or post-training, enough to know when to reach for it.

・Experience delivering with distributed teams in Japan or Asia.

#engineer #applicationsengineer #aianddatadiv

Seekless

Seek less: one search across companies' own career pages, instead of a dozen job boards.

Product

Find jobs Companies Pricing Free Resume Checker

Company

About Blog

Legal

Privacy Terms Cookies

Rakuten