[mlakolar]

Depends on CovSel's master, where mlakolar/CovarianceSelection.jl#17 (CovSel on ProximalSolvers) is merged, and on ProximalSolvers 0.1.0, formerly ProximalOPT, now mlakolar/ProximalSolvers.jl, with a new UUID. Its registration (JuliaRegistries/General#173068) merges around Oct 26. ProximalBase 0.5 is registered (0.5.0 and 0.5.1) and comes from the registry.

• [sources] takes CovSel from its master, and ProximalSolvers at 9cdd33c, the commit of its registration. That is the same in exper/ and benchmark/compare/.
• On Julia 1.10, which ignores [sources], CI adds both by URL in one Pkg.add call.

DiffPrecTest's two coordinate descent losses and its five copies of the two-stage solve are replaced by ProximalBase losses and ProximalSolvers' solver. CoordinateDescent is no longer a dependency. Version 0.3.0, since exported names are removed.

What changes

• The inverse Hessian losses are ProximalBase's KronQuadratic.
  • invAsymHessianEstimation: ⟨Θ, ΣxΘΣy⟩/2 - Θ[a, b] on vec(Θ).
  • invQSymHessian: the reduced loss θ'Hθ/2 - θ[ind] in θ = svec(Θ), which is ⟨Θ, ΣxΘΣy⟩/2 + ⟨C, Θ⟩ of the symmetric Θ. A coordinate step on Θ[r, c] is the step on θ[k], since the off-diagonal entries of svec are only scaled by √2. The weights move with that scaling, and the result is returned in svec coordinates as before.
  • CDInverseKroneckerLoss and CDInverseReducedSymKroneckerLoss, with their initialize!, gradient, descendCoordinate! and stopping scales, are deleted (about 200 lines).
• diffEstimation uses CovSel's directDifferenceLoss, and the reduced estimators reducedDiffEstimation and invHessianReduced use a QuadraticFunction(H, b).
• One driver for the two stages. Each of the five estimators solves twice, before and after recomputing its weights. The new _twoStage! builds the ProximalSolvers model once and calls solve! twice. It does not use reweight!, which would recompute the weights a third time after the last solve. For invAsymHessianEstimation the weights cost O(n p³) and take most of its time.
• Solver options. The five estimators, and supportEstimate(DTraceValidationSupport(), ...), take a ProximalSolvers CDSolver in place of CDOptions; _warmStart is removed. The defaults are the same: random order, tolerance 1e-7, scale-free stopping rule, and momentum for the quadratics as in CoordinateDescent 0.6.
• DTraceValidationSupport runs ProximalSolvers' solve_path without screening, and computes the validation losses in its callback.
• SeparateNormal fits its four node-wise lasso regressions by solve!((1/n) * LeastSquaresLoss(y, X), NormL1(λ), CDSolver()), with the residuals and their standard deviation. That is all it used of CoordinateDescent's lasso, so DiffPrecTest no longer waits for CoordinateDescent 0.7.
• Index helpers. tril_index and tril_cartesian replace sub2indLowerTriangular and ind2subLowerTriangular, also in exper/.
• The weight and variance loops compute each coordinate's entry once.
  • The weights of the reduced estimators and the variance of ReducedOracleNormal call skron and svec in their innermost loops, which ran tril_cartesian twice per term.
  • ProximalBase 0.5.0's tril_cartesian is three times slower than 0.4's ind2subLowerTriangular, which made the reduced path 1.33–1.39 times slower than master. 0.5.1 (tril_cartesian: the floating square root below p = 2^24 ProximalBase.jl#24) and #26 fix it at its source as well.
  • skron and svec now also take the entry (row, col), computed once per call. The results are bitwise identical, and the reduced path is now faster than master whichever ProximalBase it runs on.
• benchmark/compare/ (new) runs the estimators of master and of this branch on the same problems, in interleaved rounds (ab.sh).

Breaking changes (version 0.3.0)

1. CDInverseKroneckerLoss and CDInverseReducedSymKroneckerLoss are no longer exported or defined.
2. diffEstimation, invAsymHessianEstimation, invQSymHessian, reducedDiffEstimation, invHessianReduced and supportEstimate(DTraceValidationSupport(), ...) take a ProximalSolvers CDSolver as their optional last argument, in place of a CoordinateDescent CDOptions.
3. supportEstimate(DTraceValidationSupport(), ...) now throws for penalty levels that are not decreasing, which its documentation already required.
4. Inputs of the estimators are ProximalBase 0.5 iterates: a SymmetricSparseIterate is a p × p matrix in 0.5.

Benchmark against master (benchmark/compare/)

Four interleaved rounds on one core of an AMD EPYC 7513 (USC CARC Discovery, job 13326859), Julia 1.13.2, one BLAS thread, of this branch at 6fe3cf5 on ProximalOPT 1f82ae5 (now ProximalSolvers) and ProximalBase 0.5.0. The old side is master (0038089) with CovSel master, on CoordinateDescent 0.6.0 and ProximalBase 0.4.1. Times are the fastest median over the rounds, in ms; benchmark/compare/README.md has the same.

case	old	old, momentum = false	new	new / old	largest difference
diffEstimation p=30	22.1		21.5	0.97	3e-08
invAsymHessianEstimation p=30 (2, 1)	25.3		25.3	1.00	3e-08
invQSymHessian p=30 (2, 1)	1.2		0.8	0.66	3e-09
diffEstimation p=50	99.4		97.8	0.98	3e-08
invAsymHessianEstimation p=50 (2, 1)	140.1		140.7	1.00	8e-10
invQSymHessian p=50 (2, 1)	3.5		2.3	0.65	7e-09
reducedDiffEstimation p=20	198.3	200.6	91.3	0.46	1e-07
invHessianReduced p=20 (2, 1)	30.0	30.0	14.8	0.49	1e-07
reducedDiffEstimation p=30	637.2	639.9	302.9	0.48	1e-07
invHessianReduced p=30 (2, 1)	67.0	67.1	33.1	0.49	3e-08
estimate SymmetricNormal p=30 (2, 1)	113.7		114.0	1.00	0
estimate AsymmetricNormal p=30 (2, 1)	110.0		108.9	0.99	0
estimate ReducedNormal p=20 (2, 1)	266.7		123.3	0.46	0
estimate SeparateNormal p=100 (2, 1)	0.5		0.5	1.02	0
supportEstimate DTraceValidationSupport p=50 20λ	485.9		179.6	0.37	2e-07
supportEstimate ANTSupport p=20 n=200	6350.5		4348.6	0.68	0

• No slowdown against the hand-written solvers. The geometric mean of new / old is 0.69. Within each round the ratio is at most 1.04, apart from the 0.5 ms SeparateNormal case, which varies from 0.90 to 1.15 between rounds.
• diffEstimation and invAsymHessianEstimation take the same time: their O(n p³) weights take most of it. invQSymHessian takes a third less.
• The reduced estimators take half the time.
  • Their coordinate descent is about four times faster (0.12 against 0.46 ms at p = 20).
  • The rest is the weight-loop change above. Before it, the same benchmark measured them 1.33–1.39 times slower (job 13326410), because of ProximalBase 0.5.0's tril_cartesian.
  • Momentum makes no difference to them, as the old code without it shows.
• DTraceValidationSupport is almost three times faster, mostly from CovSel's diffLoss, which now multiplies dense matrices.
• The solutions agree to 2e-7, the tolerance of coordinate descent, and the refitted estimates agree exactly.
• Measured versions. The run used ProximalOPT 1f82ae5 and ProximalBase 0.5.0. This PR now takes ProximalSolvers 0.1.0 (ProximalOPT renamed, the solver code of aeed600) and ProximalBase 0.5.1, which only save time (a skipped square root for coordinates that do not move, the faster tril_cartesian).

Testing

• Pkg.test() with --depwarn=error passes all 20172 tests on Julia 1.13.2 (through [sources]) and on 1.10.12 (added by URL, as the CI step does), against ProximalSolvers 0.1.0 (9cdd33c) and ProximalBase 0.5.1, and earlier against ProximalOPT aeed600 and ProximalBase 0.5.0. Aqua is included.
• The weight-loop change leaves the results bitwise identical. That was checked on reducedDiffEstimation and invHessianReduced at p = 20 and 30, estimate(ReducedNormal(), ...) and supportEstimate(ANTSupport(), ...).
• The tests of the deleted losses' internals are replaced. Both KronQuadratic forms are checked against coordinate descent on the explicit kron(Σy, Σx) and H, the svec gradient against Hθ - e, and the scale invariance of the passes. The second stages are checked against the rows of the inverse Hessians, as before.

Release order

1. ProximalBase 0.5 is registered (0.5.0, 0.5.1).
2. ProximalSolvers 0.1.0 is registered (New package: ProximalSolvers v0.1.0 JuliaRegistries/General#173068, around Oct 26). This PR already uses the new name.
3. CovSel#17 is merged (427a2b2). CovSel's registration is postponed: it will be renamed CovarianceSelection, and registered once things settle.
4. This PR merges into master, taking CovSel from its master.
5. After the rename, DiffPrecTest moves to CovarianceSelection. Once its dependencies are registered, point [sources] at them or remove them, and remove the CI step. DiffPrecTest itself stays unregistered.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 99.27536% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.45%. Comparing base (0038089) to head (69236b6).

Files with missing lines	Patch %	Lines
src/support.jl	83.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
- Coverage   98.65%   98.45%   -0.20%     
==========================================
  Files          12       12              
  Lines        1409     1294     -115     
==========================================
- Hits         1390     1274     -116     
- Misses         19       20       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.