arXiv preprint 2026Modular agent learning
WeAct: Dynamic Weight Switching as Agent Actions
An agent can change the weights it uses as a task unfolds. WeAct trains separate capability adapters and lets the agent activate them through tool calls, continuing from a shared interaction history.
- Result
- 45.47% average accuracy across eight benchmarks, versus 42.80% for full fine-tuning.
- New vs prior
- Independently trained adapters become actions that change the agent’s active weights during a trajectory.
- Limitation
- Evaluated on one backbone with a small adapter library; sustained continual learning remains open.
Different stages need different capabilities
A deep-research agent plans its investigation, searches for evidence, checks what it found and writes an answer. These stages need different capabilities, yet conventional fine-tuning updates shared parameters and deploys a single configuration throughout the task. Learning or refreshing one capability can therefore change parameters that another capability relies on.
Independent LoRA adapters separate those updates, but merging the adapters into one fixed model can still change how each specialist behaves. WeAct keeps the adapters separate during execution as well: the agent selects the weights appropriate to the current stage through its existing tool-calling interface. The research question is whether this separation supports both effective composition and local updates to individual capabilities.
Composition improves accuracy; local updates extend the library
45.47%
Mean accuracy across eight deep-research benchmarks
WeAct-hard improves on Full SFT’s 42.80% by 2.67 percentage points, a 6.2% relative gain. WeAct-soft averages 43.21%.
+27.5%
Relative gain over the strongest static merge
WeAct-hard’s 45.47% versus TIES-Merging’s 35.66% is a 9.81-point difference. Both WeAct variants outperform every tested static merge on all eight benchmarks.
Across eight benchmarks, WeAct-hard has the highest mean accuracy at 45.47% and leads on six benchmarks. WeAct-soft averages 43.21% and leads on GAIA and MuSiQue. Full SFT averages 42.80%, Single LoRA 38.73%, and the strongest static merge, TIES-Merging, 35.66%. The headline 6.2% and 27.5% are relative improvements in the mean score; the corresponding absolute gains are 2.67 and 9.81 percentage points. The prescribed hard policy is strongest overall, so the result does not imply that flexible scheduling is always better.
Controlled diagnostics distinguish two failure modes. In sequential training, a shared rank-96 adapter still loses an average 1.67 points of teacher-forced content-token accuracy across capabilities; random partitioning loses 1.12 points. Capability-aligned adapters preserve their parameters through later stages. Separately, every tested static fusion increases mean target-token negative log-likelihood relative to the corresponding individual specialist, including full additive merging without rank compression. These are diagnostic measurements that support capability separation, rather than additional end-to-end benchmark scores.
The library can also change locally. Adding a Code adapter raises MegaScience accuracy from 26.5% to 29.0%, SciBench from 40.0% to 40.5%, and DS-1000 from 14.0% to 16.0%, with a 1.0-point gain on GAIA (Table 2). Refreshing only Writing improves ResearchQA and DeepResearch Bench under both routing policies (Table 3). The backbone and other adapters stay fixed, and neither update replays their historical training data. These studies demonstrate one capability insertion and one local refinement; they do not test a long sequence of continual updates.
Efficiency depends on the reference. WeAct uses 142.8 nominal EFLOPs for training, less than Full SFT but 1.96 times Single LoRA’s compute. In a controlled WebWalkerQA timing evaluation with 50 runs per method, soft and hard reduce mean answer-ready latency by 33.4% and 20.3% relative to Full SFT. These latency results include adapter switching and cache handling, and apply to that controlled evaluation rather than every task or deployment.
Table 1, selected methods, transposed for comparison. Accuracy (%) on the same eight benchmarks; higher is better. The average is the unweighted mean across benchmarks. Original nominal standard-error estimates are available in the linked paper. Source ↗
| Benchmark | Full SFT | Single LoRA | TIES-Merging | WeAct-hard | WeAct-soft |
|---|---|---|---|---|---|
| GAIA | 41.75 | 35.92 | 34.95 | 40.78 | 46.60 |
| 2Wiki | 50.90 | 49.70 | 47.60 | 56.30 | 53.20 |
| Bamboogle | 65.60 | 54.40 | 55.20 | 69.60 | 62.40 |
| MuSiQue | 15.76 | 15.35 | 14.07 | 17.21 | 17.58 |
| WebWalkerQA | 38.53 | 35.15 | 34.41 | 40.00 | 37.79 |
| xbench-DS | 50.00 | 51.00 | 44.00 | 54.00 | 50.00 |
| BrowseComp | 13.50 | 12.50 | 7.00 | 17.50 | 15.50 |
| FRAMES | 66.38 | 55.83 | 48.06 | 68.33 | 62.62 |
| Average | 42.80 | 38.73 | 35.66 | 45.47 | 43.21 |
Scope. The main results use Qwen3-32B and a small library of three capability adapters. The paper’s uncertainty estimates are nominal task-based binomial standard errors, not variability across repeated training runs. Keeping other parameters unchanged prevents overwriting them, but does not establish retention within an updated adapter or stable routing through many successive updates. Scaling the library and defining capability boundaries for less structured tasks remain open questions. Read the study ↗
Read the abstract
Long-horizon agents must learn heterogeneous capabilities from interleaved trajectories. Updating shared parameters to learn new capabilities can degrade previously acquired ones, resulting in catastrophic forgetting. As a trajectory unfolds, different stages may require different capabilities, yet conventional joint training and static adapter merging maintain a fixed parameter configuration throughout execution. We introduce WeAct (Weight-level Internal Actions), which trains independent capability-specific LoRA adapters over a frozen backbone and adds adapter selection to the agent's action space. These internal actions use the model's native tool-calling interface for on-demand capability composition in a shared context. Across eight deep-research benchmarks, Qwen3-32B with WeAct improves average accuracy by 6.2% relative to full fine-tuning counterparts despite using less compute, and by 27.5% relative to the strongest static LoRA-merging method. WeAct also supports modular continual adaptation, by adding a new code computation adapter to expand the agent's capabilities and updating the existing writing adapter to improve report quality. Both updates leave the backbone and other adapters unchanged, supporting modular continual adaptation in agents.
Full paper ↗Citation BibTeX
@misc{shi2026weactdynamicweightswitching,
title={WeAct: Dynamic Weight Switching as Agent Actions},
author={Rui Shi and Xunjian Yin and Jiacheng Sang and Zichuan Liu and Shuyan Zhou},
year={2026},
eprint={2610.26847},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.26847},
}