language	en
license	apache-2.0
library_name	pytorch
tags	time-series financial-forecasting ohlcv conformer foundation-model 1b-parameters quantitative-finance algorithmic-trading market-microstructure deep-learning
datasets	mito0o852/OHLCV-1m
metrics	roc-auc directional-accuracy huber-loss
pipeline_tag	time-series-forecasting

⚡ ULTRON // 1.189B Parameter Financial Conformer

A Foundation Architecture for Multi-Horizon Financial Microstructure Forecasting

Suryaansh Prithvijit Singh^1,2,* • Sonya Shelke^1,2 • Dr. Shivaji Pawar^2
1Airborne  •  ²Universal AI University
Emails: [email protected], [email protected] | [email protected], [email protected] | [email protected]
GitHub: @Ultron09 | HF: @Ultron28

---

🏛️ Executive Summary

The ULTRON Financial Conformer (1,189,135,740 parameters across 283 parameter tensors) is a billion-parameter hybrid convolution-attention foundation model pre-trained on 14 continuous calendar years (2010–2023, 168 monthly Parquet shards, 15,030 optimization steps) of 1-minute US equity OHLCV bars (mito0o852/OHLCV-1m) across 15 liquid US equities and ETFs (AAPL, AMD, AMZN, BAC, GOOGL, INTC, IWM, JPM, META, MSFT, NFLX, NVDA, QQQ, SPY, TSLA).

All empirical claims are 100% deterministic, peer-review-audited, and reproducible via logs/defensible_tier1_audit.json and logs/pretraining_telemetry_verified.json (generated by scripts/defensible_scientific_audit.py):

1. 1-Minute Base-Rate & Flat-Bar Decomposition: Across our stratified 15-ticker contiguous evaluation cohorts (2024: 8,044 windows, 2025: 8,060 windows, 2026: 8,100 windows), exact zero-return flat bars ($y_{t,1\text{m}}=0$) account for 7.86% (2024), 6.85% (2025), and 6.46% (2026) of observations, reducing strictly positive 1m returns $P(y_{t,1\text{m}} > 0)$ to 46.51%, 46.36%, and 46.15%. Under BCE training, the 1m logit centers around $\bar{z}_{1\text{m}} = -0.1627$, so uncalibrated $z > 0$ thresholding fires 0.0% of the time (53.49%–53.85% raw accuracy, identical to a FlatBar-Down heuristic). Removing this static threshold bias proves genuine discrimination: 1m All-Bar ROC-AUC of 0.5375 / 0.5262 / 0.5274, 1m Strict Non-Zero ROC-AUC (NZAUC, $y \neq 0$) of 0.5248 / 0.5138 / 0.5159, Prior-Calibrated Accuracy of 52.04% / 51.23% / 51.25%, and active two-sided 5m Strict Non-Zero Accuracy of 51.74% (2024) and 51.18% (2026).
2. Exact Global Cosine Geometry, Adam Arithmetic & Pre-2024 Crisis Forgetting: Using the exact float64 Hilbert-space Law of Cosines $\cos_{\text{exact}} = \frac{1 + r^2 - \delta_F^2}{2r}$, every module strictly satisfies the Cauchy-Schwarz lower bound $\cos_{\text{exact}} \ge \sqrt{1 - \delta_F^2}$: global Frobenius drift is 0.5439% ($\cos_{\text{exact}} = 0.99998521 \ge 0.99998521$) for Ep3 → WF-2024, 0.5725% ($\cos_{\text{exact}} = 0.99998361$) for WF-2024 → WF-2025, and 0.7860% ($\cos_{\text{exact}} = 0.99996911$) cumulatively for Ep3 → WF-2025. Decomposing drift into RMS weight norm $|\mathbf{W}|{\text{RMS}}$ and absolute RMS update $|\Delta\mathbf{W}|{\text{RMS}}$ proves that the Inception Tokenizer has the largest weight norm (0.02574 vs. 0.01432 for the backbone), yet still exhibits the highest relative drift (1.1773%) because its absolute RMS update (3.030e-4) is 4.00x larger than the backbone (7.575e-5). Direct evaluation on Pre-2024 Crisis Shards (2020-03 COVID crash, 2022-06 Fed rate shock, 2023-12; 7,896 windows) confirms zero overall forgetting (0.5130 → 0.5135 AUC; 2020-03 COVID 1m $\Delta\text{AUC} = +0.0052\ [+0.0015, +0.0084]$), while 2026 paired block-bootstrap 95% CIs confirm a significant forward 1m gain ($\Delta\text{AUC} = +0.0022\ [+0.0001, +0.0040]$).
3. Controlled Mechanistic Ablation & Denominator Proof: Using in-distribution mean-ablation and random 341ch/682ch controls, we prove that the apparent 15m peak in uncentered $|\Delta z_h|_2 / |z_h|2$ is a denominator effect (||z_15m||_RMS = 0.0405 as $\bar{z}{15\text{m}} \approx 0$, vs. 0.2041 at 1m and 0.1285 at 1d), whereas denominator-free $\text{RMS}(\Delta z_h)$ decays monotonically (0.0403 at 1m $\to$ 0.0136 at 15m $\to$ 0.0058 at 1d for K=7) and variance-normalized centered L2 sensitivity significantly exceeds random channel controls (49.90% for Meso K=15,31 vs. 30.89% for Random 682-Ch Control).
4. Non-Overlapping Execution Economics & Frozen 2024 Policy: Stepping by $S_h = \lceil \tau_h / 32 \rceil$ windows eliminates multi-period overlap, and tracking state-dependent turnover $|w_{i,k} - w_{i,k-1}|$ shows true state-hold breakeven is 1.15x–2.41x higher than isolated Gross / 2. The single frozen policy selected on 2024 (60m, τ=0.0σ) achieves +4.40 Gross Sharpe (+2.04 Net Sharpe at 1.5 bps, BE = 2.82 bps, MaxDD = 1.69%) in 2024 and +1.01 Gross Sharpe untouched in 2025, while a fixed demeaned $\tau=1.0\sigma$ rule at 3m achieves +5.38 (2024), +4.91 (2025), and +3.56 (2026) Gross Sharpe across all 3 years (BE = 0.35–0.44 bps), establishing that standalone OHLCV signals require passive maker/execution-timing deployment rather than taker spread crossing.

---

📑 Tier-1 Research Papers & Unified Monograph

This research is available both as a Unified Flagship Monograph (consolidating the entire checkpoint and log lineage into a single self-contained 16-page paper) and as three synchronized individual manuscripts authored by Suryaansh Prithvijit Singh, Sonya Shelke, and Dr. Shivaji Pawar (Airborne & Universal AI University • [email protected]):

#	Manuscript Title	Direct PDF Link	Core Empirical & Mathematical Contributions
★	The 1.189-Billion Parameter Financial Conformer: A Unified Empirical Audit of Pre-Training Scaling, Base-Rate Calibration, Continual Walk-Forward Geometry, Controlled Receptive-Field Ablation, and Non-Overlapping Execution Economics (Flagship Unified Monograph)	📄 Unified PDF (11p)	Complete self-contained 11-page monograph consolidating Architecture, Pre-Training Telemetry, 1m Base-Rate/Flat-Bar Decomposition, Exact float64 Law-of-Cosines Drift, Pre-2024 Crisis Forgetting, Controlled Mean-Ablation, and Non-Overlapping Frozen-Policy Backtesting.
1	Scaling 1.189-Billion Parameter Financial Conformers: Pre-Training Dynamics, Base-Rate Decomposition, and Non-Zero Multi-Horizon Alpha in Market Microstructure	📄 Paper 1 PDF (6p)	Exact 283-tensor architecture audit (1,189,135,740 params), 855-step pre-training telemetry (-15.2% loss variance contraction), 1m flat-bar & base-rate log-odds proof, Strict Non-Zero ROC-AUC (NZAUC), and $
2	Continual Walk-Forward Adaptation in 1.189B Financial Conformers: Exact Cosine Geometry, Adam Plasticity Arithmetic, and Pre-2024 Crisis Forgetting Audits	📄 Paper 2 PDF (4p)	Exact float64 Law-of-Cosines global cosine similarity ($\cos_{\text{exact}} \ge \sqrt{1-\delta_F^2}$), per-parameter $|\mathbf{W}|{\text{RMS}}$ vs. $|\Delta\mathbf{W}|{\text{RMS}}$ Adam arithmetic proof, Pre-2024 crisis stress tests (2020-03 COVID, 2022-06 Rates), and 2026 paired block-bootstrap 95% CIs.
3	Mechanistic Interpretability and Non-Overlapping Execution Economics in 1.189B Financial Conformers: Controlled Kernel Ablations, Denominator Decomposition, and Deflated Sharpe Audits	📄 Paper 3 PDF (5p)	In-distribution Mean-Ablation vs. Zero-Ablation with Random 341ch/682ch Controls, proof of the 15m denominator effect vs. monotonic $\text{RMS}(\Delta z_h)$, and non-overlapping contiguous backtesting with state turnover, DSR ($M=35$), and untouched Frozen 2024 Policy transfer.

---

📐 Exact Model Architecture Audit (1,189,135,740 Parameters across 283 Tensors)

Subsystem / Module	Configuration & Shapes	Exact Parameters	% of Total
Inception Conv1d Branches ($K=3,7,15,31,63,127$)	$6 \times [\text{Conv1d}(8 \to 341, K_i, s=4) + \text{bias}]$	673,134	0.057%
Inception Linear Projection & RMSNorm	$\mathbf{W}_{\text{proj}} (2046 \times 2048) + \text{RMSNorm}(2048)$	4,192,256	0.353%
20× GQA Attention (q, k, v, out)	$H_q=16, H_{kv}=8, d_{\text{head}}=128$, RoPE	251,658,240	21.163%
20× Depthwise Separable Conv Module	Pointwise GLU ($2048 \to 4096$), $\text{DWConv}_{K=7}$, Norm	251,985,920	21.191%
20× SwiGLU Feed-Forward Network	$d_{\text{model}}=2048, d_{\text{ff}}=5504$ (w_gate, w_up, w_down)	676,331,520	56.876%
20× Conformer Block RMSNorms	$3 \times \text{RMSNorm}(2048)$ per block (norm_attn, norm_conv, norm_ffn)	122,880	0.010%
Final Backbone RMSNorm & Dual MLP Heads	final_norm ($2,048$) + return_head & dir_head ($2\times 2,084,871$)	4,171,790	0.351%
Total Financial Conformer (1.189B)	20 Conformer Blocks, $L=2048 \to T=512$ Tokens (283 Tensors)	1,189,135,740	100.000%

---

🏆 Peer-Review-Hardened Empirical Results (logs/defensible_tier1_audit.json)

1. Stratified 15-Ticker OOS Evaluation of Frozen Epoch 3 (2024–2026)

Horizon ($S_h$ Step)	Year	$N_{\text{eff}}$	Flat %	Pos %	$P(z>0)$	Raw Acc	NZ Acc ($y \neq 0$)	Cal Acc	All AUC	NZ AUC	Non-Ovlp IC
1m ($S=1$)	2024	8,044	7.86%	46.51%	0.0%	53.49%	49.53%	52.04%	0.5375	0.5248	+0.0137
1m ($S=1$)	2025	8,060	6.85%	46.36%	0.0%	53.64%	50.23%	51.23%	0.5262	0.5138	+0.0210
1m ($S=1$)	2026	8,100	6.46%	46.15%	0.0%	53.85%	50.67%	51.25%	0.5274	0.5159	+0.0247
3m ($S=1$)	2024	8,044	4.00%	48.51%	27.0%	52.05%	50.80%	51.48%	0.5256	0.5185	-0.0191
3m ($S=1$)	2026	8,100	2.94%	48.27%	27.0%	52.17%	51.28%	51.09%	0.5204	0.5147	-0.0137
5m ($S=1$)	2024	8,044	3.38%	48.69%	38.8%	52.36%	51.74%	51.32%	0.5256	0.5219	+0.0150
5m ($S=1$)	2026	8,100	2.52%	48.40%	39.4%	51.64%	51.18%	51.17%	0.5166	0.5134	-0.0101
15m ($S=1$)	2024	8,044	1.60%	50.85%	62.3%	50.90%	50.97%	50.86%	0.5170	0.5157	+0.0259
60m ($S=2$)	2025	4,030	0.47%	50.30%	100.0%	50.30%	50.54%	50.56%	0.5117	0.5128	+0.0411
1d ($S=13$)	2024	627	0.20%	53.01%	100.0%	53.01%	53.11%	51.28%	0.5062	0.5063	+0.0051
1d ($S=13$)	2025	627	0.15%	52.72%	100.0%	52.72%	52.80%	50.22%	0.4994	0.4991	+0.0451

2. Exact float64 Law-of-Cosines Geometry & Adam Plasticity Hierarchy

Subsystem / Module	Params	Ep3 → WF24 $\delta_F$	WF24 → WF25 $\delta_F$	Ep3 → WF25 $\delta_F$	Ep3 → WF24 Exact $\cos$ ($\ge \sqrt{1-\delta_F^2}$)	$|\mathbf{W}|_{\text{RMS}}$	$|\Delta\mathbf{W}{24}|{\text{RMS}}$
Inception Tokenizer	4.87M	1.1773%	1.1807%	1.6172%	0.99993072 (0.99993070)	0.02574	3.030e-04
GQA Attention (L0–19)	251.66M	1.1705%	1.2528%	1.7160%	0.99993150 (0.99993149)	0.01241	1.452e-04
Directional Head	2.11M	1.0087%	0.9400%	1.4724%	0.99994962 (0.99994912)	0.01582	1.596e-04
All 20 Blocks Backbone	1180.06M	0.5291%	0.5591%	0.7671%	0.99998601 (0.99998600)	0.01432	7.575e-05
SwiGLU FFN (L0–19)	676.37M	0.3616%	0.3472%	0.4778%	0.99999347 (0.99999346)	0.01209	4.374e-05
Return Reg. Head	2.11M	0.2419%	0.2698%	0.2998%	0.99999707 (0.99999707)	0.01106	2.676e-05
Depthwise Conv (L0–19)	251.99M	0.1540%	0.1998%	0.2802%	0.99999881 (0.99999881)	0.01692	2.606e-05
Global Model (283 Tensors)	1,189.14M	0.5439%	0.5725%	0.7860%	0.99998521 (0.99998521)	0.01438	7.821e-05

3. Controlled Mechanistic Ablation & Non-Overlapping Execution Economics

---

📚 Citation & Author Information

If you utilize this foundation model, dataset pipelines, or academic papers in your quantitative research or trading algorithms, please cite:

@article{singh2026financialconformer,
  author       = {Suryaansh Prithvijit Singh and Sonya Shelke and Shivaji Pawar},
  title        = {The 1.189-Billion Parameter Financial Conformer: A Unified Empirical Audit of Pre-Training Scaling, Base-Rate Calibration, Continual Walk-Forward Geometry, Controlled Receptive-Field Ablation, and Non-Overlapping Execution Economics},
  year         = {2026},
  institution  = {Airborne and Universal AI University},
  doi          = {10.5281/zenodo.23466965},
  url          = {https://doi.org/10.5281/zenodo.23466965},
  note         = {Indexed on OpenAIRE: https://explore.openaire.eu/search/result?pid=10.5281\%2Fzenodo.23466965}
}

• Zenodo DOI: 10.5281/zenodo.23466965
• OpenAIRE Research Graph Record: explore.openaire.eu/search/result?pid=10.5281/zenodo.23466965

---

Architected by Suryaansh Prithvijit Singh, Sonya Shelke & Dr. Shivaji Pawar // Airborne & Universal AI University
Connect: GitHub (@Ultron09) • Hugging Face (@Ultron28) • [email protected]