Hi, thank you very much for the interesting paper and for open-sourcing the code. I have been going through both while taking reading notes, and a few points came up that I could not resolve on my own. I have grouped them below by theme, and if I have misread anything in the code, please do correct me. 🙏
A. Reproducing the reported results
A1. Which setting exactly produced Table 2?
The main text reports Qwen3-8B on 4×A800 (Sec. 4.1), Appendix C reports Qwen3-1.7B on a single RTX 4090D, and the only training script in the repo uses Qwen3-4B on one GPU (examples/time_series_forecast/run_qwen3-4B.sh). The GRPO settings also differ from Appendix C, which lists group size G=8 and KL coefficient 0.04, while the script sets rollout.n=6 and kl_loss_coef=0.001. In addition, some Table 2 numbers seem to match the 4B row of the scaling study (NP) and others the 8B row (PJM, ETTh, ETTm). May I ask which backbone, hyperparameters, and input setting the reported numbers come from?
A2. Were the metrics computed on the target channel only?
The paper formulates the task as multivariate (Sec. 3.1/3.4, Table 5). In the released code, however, the dataset builders serialize only the target channel into the prompt (ett_rl_dataset_builder.py: field set to "OT" or "target" for all datasets, EPF exogenous variables dropped), and the bundled checkpoints are univariate (models/patchtst/config.json: enc_in: 1, features: "S"). It would help me a lot to know whether the reported metrics were obtained on the target channel alone.
A3. Is the SFT+RL pipeline trained per dataset?
Everything released so far seems dataset-specific: the training script consumes a single train.parquet, the curriculum sampler is ETTH1-only (etth1_curriculum_sampler.py), and the reward function embeds per-dataset baselines (reward.py: RAW_BASELINE_METRICS). Does this mean the paper's results come from one agent per dataset, that is, ten separate SFT+RL runs? I would also love to know whether you tried training a single agent jointly across datasets, and how it compared.
A4. Window stride and baseline comparability
The released builders use a window stride of 96 for ETT/Wind and 48 for EPF (ett_rl_dataset_builder.py), which yields essentially non-overlapping windows and only about 126 training samples on ETTh1, far fewer than under the stride-1 windowing that most LTSF baselines use. I am concerned that the Table 2 baselines may not converge under such limited samples, if they were evaluated under the same setting.
A5. SFT artifacts
The repo contains the SFT data builders but neither the resulting ~200-trajectory dataset nor an SFT training script. Are there plans to release them? Without them, the first training stage is difficult to reproduce.
B. Points where I may be misreading the paper
B1. Long-horizon planning and iterative refinement
The paper frames forecasting as long-horizon sequential decision-making with iterative refinement (Sec. 3.1, 3.4). From the released code, however, I understand the workflow as a fixed three-turn scaffold, feature extraction, prediction, then output, where the final turn allows "at most one simple global offset or scale correction" with <think> capped at 40 tokens (prompts.py, lines 92-99; time_series_forecast_agent_flow.py). I may well be missing something here. Does the long-horizon aspect mainly live in how Turn 1 plans tool usage, or was a more open-ended refinement loop explored and found less stable? I would genuinely appreciate any pointer on how to reconcile the two.
B2. Memory compression
Sec. 3.2 and Appendix D mention that outdated memory content "may be summarized, compressed, or selectively retained". I could not locate this logic in the released code; history_analysis appears to be append-only (time_series_forecast_agent_flow.py, line 952: "Show all analyses"). Is the compression handled in a component I missed, or planned for a later release?
C. Scalability and cost
C1. High-dimensional multivariate input
The highest dimensionality in the paper is 7 (ETT, Wind). Serializing 7 channels × 96 steps as text would already multiply the prompt tokens several-fold, and the current 8192-token budget only fits a single channel. I am curious how you envision the framework handling benchmarks such as Traffic (~862 channels). Is multivariate input on the roadmap?
C2. Latency, memory, and cost-effectiveness
As I understand it, adopting the framework means training PatchTST and iTransformer separately on every dataset with TSLib (README, model_server.py), while each inference episode runs at least three LLM turns plus tool calls that include Chronos-2. Neither the paper nor the README reports latency or memory figures, although benchmark_castr1.py already measures per-episode latency (mean/p50/p95), LLM turns, and generated tokens. In your experience, how favorable is the accuracy-vs-cost trade-off compared with directly training and running the tool models? Sharing representative measurements would be very helpful for potential adopters like me.
One smaller question
On the w/o-RL ablation: since RL samples n=6 trajectories per query, the policy sees far more data than during the 200-sample SFT stage. Did you happen to run a budget-matched control that separates the gains of policy optimization from those of extra training? I ask only out of curiosity about how to interpret the gap.
(P.S. The README also refers to run_qwen3-1.7B.sh and scripts/run_ett.sh, which are not in the current tree; I assume they are coming in a later update.)
Thanks again for the nice work. These questions come from a genuine interest in reproducing and building on it, and I am happy to provide more details wherever useful.
Hi, thank you very much for the interesting paper and for open-sourcing the code. I have been going through both while taking reading notes, and a few points came up that I could not resolve on my own. I have grouped them below by theme, and if I have misread anything in the code, please do correct me. 🙏
A. Reproducing the reported results
A1. Which setting exactly produced Table 2?
The main text reports Qwen3-8B on 4×A800 (Sec. 4.1), Appendix C reports Qwen3-1.7B on a single RTX 4090D, and the only training script in the repo uses Qwen3-4B on one GPU (
examples/time_series_forecast/run_qwen3-4B.sh). The GRPO settings also differ from Appendix C, which lists group size G=8 and KL coefficient 0.04, while the script setsrollout.n=6andkl_loss_coef=0.001. In addition, some Table 2 numbers seem to match the 4B row of the scaling study (NP) and others the 8B row (PJM, ETTh, ETTm). May I ask which backbone, hyperparameters, and input setting the reported numbers come from?A2. Were the metrics computed on the target channel only?
The paper formulates the task as multivariate (Sec. 3.1/3.4, Table 5). In the released code, however, the dataset builders serialize only the target channel into the prompt (
ett_rl_dataset_builder.py:fieldset to "OT" or "target" for all datasets, EPF exogenous variables dropped), and the bundled checkpoints are univariate (models/patchtst/config.json:enc_in: 1,features: "S"). It would help me a lot to know whether the reported metrics were obtained on the target channel alone.A3. Is the SFT+RL pipeline trained per dataset?
Everything released so far seems dataset-specific: the training script consumes a single
train.parquet, the curriculum sampler is ETTH1-only (etth1_curriculum_sampler.py), and the reward function embeds per-dataset baselines (reward.py: RAW_BASELINE_METRICS). Does this mean the paper's results come from one agent per dataset, that is, ten separate SFT+RL runs? I would also love to know whether you tried training a single agent jointly across datasets, and how it compared.A4. Window stride and baseline comparability
The released builders use a window stride of 96 for ETT/Wind and 48 for EPF (
ett_rl_dataset_builder.py), which yields essentially non-overlapping windows and only about 126 training samples on ETTh1, far fewer than under the stride-1 windowing that most LTSF baselines use. I am concerned that the Table 2 baselines may not converge under such limited samples, if they were evaluated under the same setting.A5. SFT artifacts
The repo contains the SFT data builders but neither the resulting ~200-trajectory dataset nor an SFT training script. Are there plans to release them? Without them, the first training stage is difficult to reproduce.
B. Points where I may be misreading the paper
B1. Long-horizon planning and iterative refinement
The paper frames forecasting as long-horizon sequential decision-making with iterative refinement (Sec. 3.1, 3.4). From the released code, however, I understand the workflow as a fixed three-turn scaffold, feature extraction, prediction, then output, where the final turn allows "at most one simple global offset or scale correction" with
<think>capped at 40 tokens (prompts.py, lines 92-99;time_series_forecast_agent_flow.py). I may well be missing something here. Does the long-horizon aspect mainly live in how Turn 1 plans tool usage, or was a more open-ended refinement loop explored and found less stable? I would genuinely appreciate any pointer on how to reconcile the two.B2. Memory compression
Sec. 3.2 and Appendix D mention that outdated memory content "may be summarized, compressed, or selectively retained". I could not locate this logic in the released code;
history_analysisappears to be append-only (time_series_forecast_agent_flow.py, line 952: "Show all analyses"). Is the compression handled in a component I missed, or planned for a later release?C. Scalability and cost
C1. High-dimensional multivariate input
The highest dimensionality in the paper is 7 (ETT, Wind). Serializing 7 channels × 96 steps as text would already multiply the prompt tokens several-fold, and the current 8192-token budget only fits a single channel. I am curious how you envision the framework handling benchmarks such as Traffic (~862 channels). Is multivariate input on the roadmap?
C2. Latency, memory, and cost-effectiveness
As I understand it, adopting the framework means training PatchTST and iTransformer separately on every dataset with TSLib (README,
model_server.py), while each inference episode runs at least three LLM turns plus tool calls that include Chronos-2. Neither the paper nor the README reports latency or memory figures, althoughbenchmark_castr1.pyalready measures per-episode latency (mean/p50/p95), LLM turns, and generated tokens. In your experience, how favorable is the accuracy-vs-cost trade-off compared with directly training and running the tool models? Sharing representative measurements would be very helpful for potential adopters like me.One smaller question
On the w/o-RL ablation: since RL samples n=6 trajectories per query, the policy sees far more data than during the 200-sample SFT stage. Did you happen to run a budget-matched control that separates the gains of policy optimization from those of extra training? I ask only out of curiosity about how to interpret the gap.
(P.S. The README also refers to
run_qwen3-1.7B.shandscripts/run_ett.sh, which are not in the current tree; I assume they are coming in a later update.)Thanks again for the nice work. These questions come from a genuine interest in reproducing and building on it, and I am happy to provide more details wherever useful.