• MDPs & value methods: State/action spaces, Q-learning, DQN, Double DQN, Dueling DQN
• Policy gradient methods: REINFORCE, Actor-Critic, A2C/A3C, variance reduction
• Advanced optimization: PPO, TRPO, SAC, trust regions, entropy regularization
• TD learning: TD(0), TD(λ), eligibility traces, bootstrapping methods
LLM Alignment & Post-Training
• RLHF pipelines: Reward model training, preference learning, human feedback integration
• Direct optimization: DPO, IPO, KTO, offline preference optimization
• Group-based methods: GRPO, RLOO, sample-efficient policy improvement
• Reward modeling: Bradley-Terry models, reward hacking mitigation, KL constraints
• Gymnasium/OpenAI Gym: Custom environments, observation/action spaces, wrapper patterns
• Reward engineering: Sparse vs. dense rewards, potential-based shaping, intrinsic motivation
• Verifier design: Programmatic reward functions, outcome verification, ground-truth evaluation
• Simulation: Sim-to-real transfer, domain randomization, multi-agent dynamics
• Offline RL: CQL, BCQ, IQL for learning from fixed datasets without environment interaction
• Model-based RL: World models, Dreamer, MuZero, learned dynamics
• Hierarchical RL: Options framework, goal-conditioned policies, temporal abstraction
• Imitation & exploration: Behavioral cloning, GAIL, curiosity-driven exploration, UCB