📋 摘要
⭐ LLM解码采样方法，与SE for AI、测试与公平性等核心兴趣关联较弱，仅在代码生成基准上略有擦边。
Large Language Models Explore by Latent Distilling
中文
本文针对大语言模型 (LLM) test-time scaling 中样本多样性不足的问题：标准 stochastic sampling 主要产生表层词汇变化，难以实现语义层面的探索。作者提出 Exploratory Sampling (ESamp)，一种显式鼓励语义多样性的解码方法。其思路基于神经网络对熟悉输入预测误差较低、对新颖输入预测误差较高的性质：在 test time 训练一个轻量级 Distiller，从 LLM 的浅层 hidden representation 预测深层 hidden representation，以建模其 depth-wise 表征转移；解码过程中 Distiller 持续适配当前生成上下文所诱导的映射，并将其预测误差作为 novelty 信号，对候选 token 扩展进行重加权，从而将解码偏向探索较少的语义模式。方法采用异步训练-推理流水线，最坏情况开销低于 5% (优化版本仅 1.2%)。实验表明，ESamp 显著提升 reasoning model 的 Pass@k 效率，在数学、科学和代码生成 benchmark 上相较强 stochastic 与 heuristic baselines 取得更优或相当的表现，并在 creative writing 中突破多样性与连贯性之间的 trade-off，展现良好的泛化性。
English abstract
Generating diverse responses is crucial for test-time scaling of large language models (LLMs), yet standard stochastic sampling mostly yields surface-level lexical variation, limiting semantic exploration. In this paper, we propose Exploratory Sampling (ESamp), a decoding approach that explicitly encourages semantic diversity during generation. ESamp is motivated by the well-known observation that neural networks tend to make lower-error predictions on inputs similar to those encountered before, and incur higher prediction error on novel ones. Building on this property, we train a lightweight Distiller at test time to predict deep-layer hidden representations of the LLM from its shallow-layer representations to model the LLM's depth-wise representation transitions. During decoding, the Distiller continuously adapts to the mappings induced by the current generation context. ESamp uses the prediction error as a novelty signal to reweight candidate token extensions conditioned on the current prefix, thereby biasing decoding toward less-explored semantic patterns. ESamp is implemented with an asynchronous training--inference pipeline, with less than 5% worst case overhead (1.2% in the optimized release). Empirical results show that ESamp significantly boosts the Pass@k efficiency of reasoning models, showing superior or comparable performance to strong stochastic and heuristic baselines. Notably, ESamp achieves robust generalization across mathematics, science, and code generation benchmarks and breaks the trade-off between diversity and coherence in creative writing. Our code has released at: https://github.com/LinesHogan/tLLM.
加载中…
点文件 → 加为 tab；按 Esc 关闭