👍 90
09/25 20:00
Reinforcement learning (RL) for code agents often uses executable tests to provide binary rewards. With these rewards, Group Relative Policy Optimization (GRPO) assigns identical advantages to test-passing trajectories within each rollout group, overlooking differences in implementation quality and
中文介绍 提出了一种改进的强化学习方法,旨在优化代码代理的评估和奖励分配。该方法通过 Group Relative Policy Optimization (GRPO) 重新分配测试通过轨迹的优势,对实现质量的差异进行了更深入的考量,以便更准确地评估代理在多种环境中的表现。这一新方法能够提高学习效率和效果,对于代码代理的可靠性和表现评估有重要影响。意义:增强了代码代理在复杂任务中的表现评估能力。
👍 55
09/27 20:00
Most modern multimodal large language models (MLLMs) build on a pretrained visual encoder that provides a strong visual prior. Encoder-free MLLMs instead learn visual representations directly from raw pixels, offering a simple and unified architecture, but their scaling behavior has not been systema
中文介绍 探讨了无编码器的多模态大模型(MLLM)在视觉表示学习方面的表现,旨在消除传统视觉编码器的依赖。通过直接从原始像素学习图像特征,本研究提出了一种简单统一的模型架构。分析表明,随着规模增大,这种无编码器方法的表现趋向于可靠,但其扩展行为需要系统性研究。意义:推动多模态学习中对编码器依赖的减少,对图像理解和处理具有深远影响。
👍 29
09/26 20:00
A central goal in robotics is to enable manipulation across changing tasks and environments. Vision-language-action (VLA) models provide broad manipulation capabilities but can struggle when execution requires diagnosing failures and adapting behavior. Strong agents can discover effective interventi
中文介绍 提出了一种递归式的分布式蒸馏方法以改进机器人操作,针对多变的任务和环境进行适应。结合视觉-语言-行动模型,该方法使得机器人能够在处理失败时进行行为调整。研究表明,强大的代理可以有效地发现并实施干预措施,以应对复杂的任务挑战,提高了机器人适应不同环境的能力。意义:有助于机器人在动态环境中的操作灵活性与智能化发展。
👍 27
09/26 20:00
Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of model--environment interactions, and nearly 1M tokens per rollout. Applying online reinforcement learning (RL) to such executions poses two fundamental ch
中文介绍 提出了一种弹性强化学习框架 QwenGyre,专为处理极长时间跨度的任务而设计。该框架优化了在线强化学习(RL)在高延迟任务中的效率,处理了模型与环境之间的多次互动以完成复杂任务。研究结果表明,此框架在长时间任务上显著提升了学习效率和任务表现,解决了长期任务中的关键挑战。意义:为极长时间跨度任务的智能代理开发提供了新工具。
👍 25
09/27 20:00
Vision-language (VL) pretraining using paired chest X-ray (CXR) images and radiology reports has shown strong potential for medical image understanding. However, existing methods often remain dependent on task-specific finetuning because radiology reports are lengthy, clinically dense, and difficult
中文介绍 提出了一种以句子为中心的增强视觉-语言预训练方法 SentZero,专注于多任务零-shot的胸部 X 光图像分析。新方法通过优化视觉表示与医学报告的配对来提高模型的理解能力,以克服现有方法在任务特定微调中的限制。结果显示,该方法在多个医学影像任务上表现了更强的泛化能力。意义:推动了医学影像处理领域的多任务学习发展。
👍 17
09/25 20:00
Fast matrix multiplication algorithms keep the product fixed and search for a cheaper way to evaluate it. We instead ask whether a Transformer's learned projections can use a different, cheaper product altogether. Building on an associative-algebra construction that replaces ordinary matrix multipli
中文介绍 通过引入关联代数层,研究了在 Transformer 中以便宜的方式替代传统矩阵乘法的可能性。该方法借助联接代数的构建,探索了如何在保持参数不变的情况下改变产品。在实验中,该方法展示了增强运算效率的潜力,并为深度学习模型的架构设计提供了新视角。意义:为模型运算优化与资源节约的研究提供了新的思路。
👍 16
09/26 20:00
Coding-agent evaluation has progressed from resolving individual issues to carrying out long-horizon development, yet task completion is still largely assessed within a single codebase. In software ecosystems, many features and bug fixes require coordinated changes across multiple repositories. We i
中文介绍 针对编程代理的长远开发任务,提出了一种新方法 WideSWE,旨在解决多代码库间的协调变更。研究展示了在软件生态系统中,许多特性和修复需跨多个代码仓库协作完成,当前评估方法过于局限于单一代码库。通过提出新评估标准,该研究有助于提升跨代码库开发的效率。意义:增强了编程代理在复杂软件开发环境中的能力。
👍 15
09/27 20:00
Frontier general-purpose systems are rapidly expanding beyond visual understanding into capabilities traditionally handled by dedicated computer-vision models. As these capabilities expand, a central question for the computer-vision community is how far this reach extends, and what remains hard. We
中文介绍 研究了前沿通用系统在计算机视觉能力上的扩展,探讨其与传统专用视觉模型之间的区别。随着能力的扩展,该研究旨在识别难度较大的视觉任务,并解析新系统在视觉理解方面的局限性。这些发现对计算机视觉领域的模型开发与优化策略具有重要启示。意义:推动计算机视觉研究者对新兴系统的认识与应用。
👍 15
09/27 20:00
Tool-based image editing (image retouching) is commonly formulated with autoregressive multimodal large language models (MLLMs) that sequentially generate reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a
中文介绍 提出了一种新颖的图像 retouching 方法 FlowTool,通过流匹配控制工具参数来实现基于工具的图像编辑。该方法重新构建了图像编辑任务的框架,并通过自回归多模态大模型有效整合推理、工具选择和参数设置。研究结果显示,该方法在图像处理的灵活性与效率上有显著提升,推动了图像编辑技术的发展。意义:提升了图像编辑的智能化和自动化水平。
👍 11
09/27 20:00
As language models take a growing role in AI development, a natural aspiration is for them to reflect on their own learning process, as humans do, and use that reflection to improve themselves. At the same time, these models have an advantage that human learners lack, since training leaves parameter
中文介绍 探讨了语言模型在自我学习和自我改进上的潜力,考虑其反思学习过程的能力。作者提出,语言模型能够以独特的方式进行行为干预,从而优化学习效果。该研究为未来的智能体发展提供了新的视角,强调自我反思在 AI 训练中的价值。意义:推动智能体在学习过程中的自我改进策略的研究。
👍 10
09/27 20:00
We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimization. Building on this connection, we introduce Least-Square Policy Distillation (LSPD), an RL-inspired framework that
中文介绍 通过强化学习的视角,研究了基于逆 KL 目标的在线策略蒸馏(OPD)。提出的最小二乘策略蒸馏(LSPD)框架连接了 OPD 和 KL 规则化策略优化,表明该框架能够有效提高模型的蒸馏效率并改善推理性能。实验验证了 LSPD 在样本效率和准确性上的提升,对 LLM 推理有重要贡献。意义:推进 LLM 领域中强化学习与蒸馏技术的结合。
👍 9
09/27 20:00
Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. As a c
中文介绍 提出了一种适应性并行化策略 Nereus,用于 LLM 的强化学习后训练,针对 GPU 集群的动态环境进行优化。随着资源、序列长度和内存压力的变化,该策略能够提高模型在生成、推理和训练中的协调性,显著提升了后训练的效率。该研究对 LLM 的部署与协同处理具有重要意义。意义:改善了 LLM 在多变环境下的适应能力与资源利用率。
👍 8
09/25 20:00
Learning from experience in LLM agents has become a key paradigm for developing self-evolving agents that continuously learn and expand their capabilities. Within this paradigm, synthesizing the agent skill has emerged as a promising solution for transforming accumulated experience into reusable pro
中文介绍 介绍了一种名为 ExpVoyager 的方法,旨在通过直接经验导航促进动态代理技能的合成。该方法强调从经验中学习,以便开发自我演变的代理,通过合成技能将积累的经验转化为可重用的能力。这种方法对智能代理的自我演化与能力提升呈现了可观潜力。意义:为动态能力发展与自我演化代理提供了新方法。
👍 5
09/25 20:00
Multiple agents may often conflict in an organization: for example, one coding agent changes an interface in a repository, but another continues to develop on the old version where existing tests become stale. A conversation can resolve the episode, but when the participants change, what makes the l
中文介绍 研究了多代理协作中的持久组织能力,探讨不同代理之间如果相互协调以确保持续的任务执行。问题在于当参与者变化时,如何维护协作一致性。通过提供实证分析,该研究为多代理环境中的有效合作提供了理论支持与实际示例。意义:促进了多代理系统中组织与协调能力的发展研究。
👍 4
09/27 20:00
Multi-vector retrievers built on vision-language models lead visual document retrieval (VDR), but they run a multi-billion-parameter query encoder on every search. Distilling this encoder into a small student that queries the teacher's existing index would remove the bottleneck. The standard recipe,
中文介绍 提出了一种无文档查询蒸馏技术 ColNanoVDR,旨在通过最优运输方法优化基于视觉的文档检索。该方法通过蒸馏大规模多向量检索模型的查询编码器,将查询体积缩减至小型学生模型,解决了传统方法中的瓶颈问题。实验证明,这一方法在检索效率和效果方面均有明显改进。意义:提升了视觉文档检索的效率与可扩展性。
👍 4
09/27 20:00
Reliable AI safeguards require both control mechanisms that reduce unsafe behavior and monitoring mechanisms that detect safety risks during model interactions. Established behavioral safeguards include alignment methods that optimize model outputs and text monitors that assess interaction text. Rep
中文介绍 探讨了在语言模型安全性方面,表征工程的作用与影响。该研究系统性地分析了可靠 AI 安全机制的构建,强调了控制机制与监控策略对于降低不安全行为的重要性,提出了一种新的安全评估方法。研究的结果对未来语言模型的安全设计与应用具有重要指导意义。意义:推动了 AI 安全性研究关注点的转变与深化。
👍 3
09/27 20:00
RLVR provides reliable trajectory-level credit, while OPSD offers dense supervision for token-level credit. This exposes a fundamental coupling when updating step-level credit direction and magnitude with teacher supervision, preventing steps from receiving reliable credit directions and contributio
中文介绍 针对自我蒸馏中的信贷方向与幅度解耦问题进行了探讨,提出了一种新方法。该方法通过 RLVR 和 OPSD 的结合,优化了如何在教师监督下更新步骤级信贷,从而提高了蒸馏过程中信贷的可靠性与贡献度。研究表明,新的信贷机制增强了模型自我蒸馏的效率。意义:推动了自我蒸馏在强化学习中的应用与理解。
👍 3
09/26 20:00
Connectionist temporal classification (CTC) naturally supports offline and streaming speech recognition with utterance-level supervision, but conventional implementations materialize frame-by-vocabulary activations in memory, making CTC training with native LLM vocabularies prohibitively memory-inte
中文介绍 提出了一种修剪连接时序分类(CTC)方法,以提高大词汇自动语音识别(ASR)训练的记忆效率。通过优化传统 CTC 的实现,避免了在内存中生成大规模词汇的激活,从而显著提升了训练效率。这一方法为降低训练资源消耗与提升技术应用场景的可行性提供了新的路径。意义:推动了大词汇自动语音识别领域的资源优化研究。
👍 3
09/27 20:00
Distillation attacks copy the reasoning capabilities of closed-source large language models, allowing bad actors to replicate state-of-the-art performance at low cost. Attackers systematically collect a large volume of frontier model reasoning traces and then train (i.e., "distill") their own models
中文介绍 研究了强化学习后蒸馏防御的有效性,发现现有的蒸馏攻击能够迅速复制封闭源大型语言模型的推理能力。该研究揭示了攻击者如何收集模型推理痕迹并进行训练,从而导致蒸馏防御失效。通过详细分析,提供了对未来安全性研究的重要见解,强调了模型开发中的保护措施。意义:为强化学习与安全性研究中的模型防御策略提供了反馈。
👍 3
09/27 20:00
Lookup-based memory has been a promising way to scale the parameters of large language models (LLMs). It retrieves learned representations of local token patterns, such as n-grams, instead of reconstructing them through successive layers of computation. However, existing designs such as Engram treat
中文介绍 提出了一种因子化 n-gram 记忆机制 FactorEngram,旨在提升语言模型的参数规模。通过检索局部标记模式的学习表示,该机制优化了参数处理过程,提高了模型在大规模数据下的有效性。研究结果证明,该方法有效缩减了模型计算复杂性,为语言模型的高效训练与推理提供了新思路。意义:促进了语言模型记忆力和参数效率的提升。