Nonlinear Solvers

Goal

Provide a universal workflow to select a nonlinear solver, configure globalization strategies, and diagnose convergence for root-finding, optimization, and least-squares problems.

Requirements

• Python 3.8+

• NumPy (for Jacobian diagnostics)

• SciPy (optional, for advanced analysis)

Inputs to Gather

Input Description Example

Problem type Root-finding, optimization, least-squares root-finding

Problem size Number of unknowns n = 10000

Jacobian availability Analytic, finite-diff, unavailable analytic

Jacobian cost Cheap or expensive to compute expensive

Constraints None, bounds, equality, inequality none

Smoothness Is objective/residual smooth? yes

Residual history Sequence of residual norms 1,0.1,0.01,...

Decision Guidance

Solver Selection Flowchart

Is Jacobian available and cheap? ├── YES → Problem size? │ ├── Small (n < 1000) → Newton (full) │ └── Large (n ≥ 1000) → Newton-Krylov └── NO → Is objective smooth? ├── YES → Memory limited? │ ├── YES → L-BFGS or Broyden │ └── NO → BFGS └── NO → Anderson acceleration or Picard

Quick Reference

Problem Type First Choice Alternative Globalization

Small root-finding Newton Broyden Line search

Large root-finding Newton-Krylov Anderson Trust region

Optimization L-BFGS BFGS Wolfe line search

Least-squares Levenberg-Marquardt Gauss-Newton Trust region

Bound constrained L-BFGS-B Trust-region reflective Projected

Script Outputs (JSON Fields)

Script Key Outputs

scripts/solver_selector.py

recommended , alternatives , notes

scripts/convergence_analyzer.py

converged , convergence_type , estimated_rate , diagnosis

scripts/jacobian_diagnostics.py

condition_number , jacobian_quality , rank_deficient

scripts/globalization_advisor.py

strategy , line_search_type , trust_region_type , parameters

scripts/residual_monitor.py

patterns_detected , alerts , recommendations

scripts/step_quality.py

ratio , step_quality , accept_step , trust_radius_action

Workflow

• Characterize problem - Identify type, size, Jacobian availability

• Select solver - Run scripts/solver_selector.py

• Choose globalization - Run scripts/globalization_advisor.py

• Analyze Jacobian - If available, run scripts/jacobian_diagnostics.py

• Monitor residuals - During solve, use scripts/residual_monitor.py

• Analyze convergence - Run scripts/convergence_analyzer.py

• Evaluate steps - For trust region, use scripts/step_quality.py

Conversational Workflow Example

User: My Newton solver for a phase-field simulation is converging very slowly. After 50 iterations, the residual only dropped from 1 to 0.1.

Agent workflow:

• Analyze convergence: python3 scripts/convergence_analyzer.py --residuals 1,0.8,0.6,0.5,0.4,0.3,0.2,0.15,0.12,0.1 --json

• Check globalization strategy: python3 scripts/globalization_advisor.py --problem-type root-finding --jacobian-quality ill-conditioned --previous-failures 0 --json

• Recommend: Switch to trust region with Levenberg-Marquardt regularization, or use Newton-Krylov with better preconditioning.

Pre-Solve Checklist

• Confirm problem type (root-finding, optimization, least-squares)

• Assess Jacobian availability and cost

• Check initial guess quality

• Set appropriate tolerances

• Choose globalization strategy

• Prepare to monitor convergence

CLI Examples

Select solver for large unconstrained optimization

python3 scripts/solver_selector.py --size 50000 --smooth --memory-limited --json

Analyze convergence from residual history

python3 scripts/convergence_analyzer.py --residuals 1,0.1,0.01,0.001,0.0001 --tolerance 1e-6 --json

Diagnose Jacobian quality

python3 scripts/jacobian_diagnostics.py --matrix jacobian.txt --json

Get globalization recommendation

python3 scripts/globalization_advisor.py --problem-type optimization --jacobian-quality good --json

Monitor residual patterns

python3 scripts/residual_monitor.py --residuals 1,0.8,0.9,0.7,0.75,0.6 --target-tolerance 1e-8 --json

Evaluate step quality for trust region

python3 scripts/step_quality.py --predicted-reduction 0.5 --actual-reduction 0.4 --step-norm 0.8 --gradient-norm 1.0 --trust-radius 1.0 --json

Error Handling

Error Cause Resolution

problem_size must be positive

Invalid size Check problem dimension

constraint_type must be one of...

Unknown constraint Use: none, bound, equality, inequality

residuals must be non-negative

Invalid residual data Check residual computation

Matrix file not found

Invalid path Verify Jacobian file exists

Interpretation Guidance

Convergence Type

Type Meaning Action

quadratic Optimal Newton Continue, near solution

superlinear Quasi-Newton working Monitor for stagnation

linear Acceptable May improve with preconditioner

sublinear Too slow Change method or formulation

stagnated No progress Check Jacobian, preconditioner

diverged Increasing residual Add globalization, check Jacobian

Jacobian Quality

Quality Condition Number Action

good < 10⁶ Standard Newton works

moderately-conditioned 10⁶ - 10¹⁰ Consider scaling

ill-conditioned

10¹⁰ Use regularization

near-singular ∞ Reformulate or use LM

Step Quality (Trust Region)

Ratio ρ Quality Trust Radius

ρ < 0 very_poor Shrink aggressively

ρ < 0.25 marginal Shrink

0.25 ≤ ρ < 0.75 good Maintain

ρ ≥ 0.75 excellent Expand if at boundary

Limitations

• No global convergence guarantee: All methods may fail for pathological problems

• Jacobian accuracy: Finite-difference Jacobian may be inaccurate near discontinuities

• Large dense problems: May require specialized solvers not covered here

• Constrained optimization: Complex constraints need SQP or interior point methods

References

• references/solver_decision_tree.md

• Problem-based solver selection

• references/method_catalog.md

• Method details and parameters

• references/convergence_diagnostics.md

• Diagnosing convergence issues

• references/globalization_strategies.md

• Line search and trust region

Version History

• v1.0.0 : Initial release with 6 analysis scripts