Use when fitting equations to data with PySR or SymbolicRegression.jl, when a user wants an interpretable formula, symbolic model, scaling law, or empirical relation discovered from numeric data, or when debugging a PySR search that is slow, stuck, or giving poor equations.
72
89%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Passed
No findings from the security scan
PySR evolves expression trees through the Julia backend SymbolicRegression.jl, returning readable equations on an accuracy/complexity Pareto front. This guide distills the documentation and several hundred real user threads. Full docs: https://ai.damtp.cam.ac.uk/pysr/
# pip install pysr (Julia is downloaded automatically on first import; no separate install)
import numpy as np
from pysr import PySRRegressor
X = 2 * np.random.randn(100, 5)
y = 2 * np.cos(X[:, 3]) + X[:, 0] ** 2 - 2
model = PySRRegressor(
binary_operators=["+", "-", "*", "/"],
unary_operators=["cos"],
niterations=100,
)
model.fit(X, y)
print(model) # Pareto front: complexity, loss, equation
model.predict(X) # uses the auto-selected "best" equation
model.sympy() # SymPy expression
model.latex() # LaTeX stringmodel.equations_ is a pandas DataFrame containing complexity, loss, score, equation, sympy_format, lambda_format. Select any front row by index in predict, sympy, latex, jax, pytorch; without an index, methods use the auto-selected equation. Indexing lets you inspect and export a specific Pareto candidate instead of only the default row.
The first .fit() pays Julia startup/JIT, roughly 1-2 minutes; subsequent same-process fits start almost instantly. Repeated fresh processes repay startup and compilation for every experiment, a common agent failure.
PySRRegressor instances are cheap; Julia runtime state lives per-process.python script.py. Process reuse is an optimization rather than a requirement.pysr; fresh environments re-resolve and precompile Julia packages.guessesInject knowledge through modifiable/discardable guesses or mandatory template structure. Combine both using component/parameter guesses. Constructor strings require Julia syntax, input variable names, and enabled operators:
model = PySRRegressor(
operators={2: ["+", "*"]},
guesses=["x0 + 0.5 * x1", "x0 * (x1 + 1.0)"],
)Single output: candidate list. Multiple outputs: lists in target-column order, guesses=[["x0 + x1"], ["x0 * x1"]]. Templates: dictionaries keyed by component/parameter names; #1, #2, etc. identify component arguments.
Guesses enter populations throughout search. should_optimize_constants=True optimizes constants/template parameters before insertion. fraction_replaced_guesses, default 0.001, controls the cycle-end replacement fraction.
Guesses are pushed into the hall of fame before generation one. Inspect accepted seeds at their complexity in model.equations_ or immediate hall_of_fame.csv; confirm insertion and optimized constants. Oversize seeds produce only Julia warning Guess expression '...' has complexity 23 > maxsize (10), using default x1, x2, ... names regardless of supplied names. They remain unrecorded but still migrate; descendants become recordable after mutations satisfy maxsize. Violations of constraints, nested_constraints, or maxdepth disappear silently. Lower-loss candidates replace seeds at the same complexity: only one expression survives per complexity.
Keep constraints compatible and set maxsize slightly above the largest seed; use the warning's complexity. Cost rises steeply, PySR warns above 40, and several-times-seed sizes can exhaust the budget without completing one generation despite appearing alive. Use warmup_maxsize_by against early bloat.
Set guesses on the estimator, never .fit(). Continue with model.set_params(guesses=[...], warm_start=True), replacing guesses while keeping data representation/search space fixed.
batching=False forces full-data search evaluations. Batching still reevaluates hall-of-fame candidates fully. Fewer rows proportionally accelerate search.pow/square/cube or -/neg enlarge search. Polynomials: ["+", "-", "*"], omit / and ^.ncycles_per_iteration were tuned by large-scale search in 2024-2025. Avoid older forum/paper recipes.niterations; control wall-clock with timeout_in_seconds, total evaluations with max_evals, or stopping with early_stop_condition="stop_if(loss, complexity) = loss < 1e-6 && complexity < 10" (Julia (loss, complexity) function). max_evals supports compute-matched comparisons; niterations * populations * population_size does not count evaluations. Monitor partial outputs/<run_id>/hall_of_fame.csv; stop gracefully with q then Enter in IPython.model_selection="best" maximizes score within 1.5x minimum loss; "accuracy" minimizes loss. Score is negative log-loss slope per complexity unit; a large score marks a sharp accuracy gain for little added complexity. These heuristics control default predict.
Print model.equations_; inspect sharp loss drops, held-out model.predict(X_test, index=i) when overfitting is plausible, and limits x -> 0/inf against domain expectations. Present 2-3 candidates with tradeoffs.
Default: MSE.
elementwise_loss="L1DistLoss()", median-seeking versus mean-seeking.model.fit(X, y, weights=1/sigma**2); built-ins apply weights automatically. Custom: elementwise_loss="myloss(x, y, w) = w * abs(x - y)^2".elementwise_loss = """function loss_fnc(prediction, target)
scatter_loss = abs(log((abs(prediction)+1e-20) / (abs(target)+1e-20)))
sign_loss = 10 * (sign(prediction) - sign(target))^2
return scatter_loss + sign_loss
end""""HuberLoss(1.0)", "LPDistLoss{3}()"."L2MarginLoss()"; apply sigmoid yourself for probabilities.weights, usually preferable to custom objectives. Strict enforcement: estimate limits numerically inside loss_function, add graded penalties.loss_scale="linear" permits negatives, including log-likelihoods.elementwise_loss receives scalar (prediction, target) or (prediction, target, weight); never sum/broadcast. Whole-vector/tree objectives require loss_function.
maxsize and rows; search selects features reasonably well, automatic batching handles volume. Requiring all 30+ features needs maxsize well above 100.select_k_features=k.Regress u_t = f(u, u_x, u_xx, ...) from u(x, t): grid points become samples, field/spatial derivatives become features, time derivative becomes target; PySR discovers products like u*u_x without a fixed library.
savgol_filter(U, 21, 3, deriv=1, delta=dx, axis=-1); finite differences require clean data, spectral derivatives catastrophically amplify noise.complexity_of_variables=[1, 2, 3, ...], costing u, u_x, u_xx as 1, 2, 3; prune dimensional impossibilities with X_units=["m/s", "s^-1", ...]. Inspect the front elbow; noisy coefficients are approximate. Validate forward simulation from held-out initial conditions.Default to plain search; suggest TemplateExpressionSpec for known structure that free-form trees would rediscover or violate:
sin(f(x1, x2)) + g(x3), rational functions, envelopes x*(1-x)*g(x).parameters; shared subexpressions/coupled outputs; derivative/integral relations through differential operator D; hard variable-placement requirements.from pysr import PySRRegressor, TemplateExpressionSpec
spec = TemplateExpressionSpec(
combine="sin(f(x1, x2)) + g(x3)^2",
expressions=["f", "g"],
variable_names=["x1", "x2", "x3"],
)
model = PySRRegressor(expression_spec=spec, binary_operators=["+", "-", "*", "/"])
model.fit(X, y)Pass category as an X column; increment zero-based IDs for Julia's 1-based indexing:
spec = TemplateExpressionSpec(
combine="p[class] * f(x1, x2) + q[class]",
expressions=["f"],
variable_names=["x1", "x2", "class"],
parameters={"p": 3, "q": 3},
)
model = PySRRegressor(
expression_spec=spec,
operators={2: ["+", "*"]},
guesses=[{
"f": "#1 + #2",
"p": [5.0, 10.0, 0.8],
"q": [0.0, 0.0, 0.0],
}],
)parameters={"p": 3, "q": 3} declares vector lengths; guesses initialize vectors and f. Declare disjoint component/parameter names and every guess key; supply every component formula. Match parameter lengths using Python lists/1D NumPy arrays; omit vectors for normal initialization. Values remain learnable, are copied without modifying caller arrays, and use model precision. Parameter guesses require SymbolicRegression.jl ≥2.3.0. #1/#2 denote passed arguments, here x1/x2; with combine="f(x, p[1])", #2 denotes p[1].
combine accepts arbitrary Julia/multiple statements: reuse fx = f(x); fx + fx^2, change arguments f(x1) - f(x2), differentiate df = D(f, 1); df(x). Vector/multi-output problems: append extra targets to X, return per-row residuals, fit dummy y with elementwise_loss="(p, t) -> p" so output itself is loss.
PySR 2.0 caveats: arbitrary Julia prevents .sympy(), .latex(), .jax(), .pytorch() export; manually reassemble component strings from model.equations_. Arguments print positionally as #1, #2 because components may receive different inputs. Combine values are ValidVectors (.x data, .valid flag); arithmetic propagates validity, custom manipulation must unwrap/rebuild ValidVector(raw, valid). Use Float32-safe 0.5f0 or explicit conversion; bare Float64 0.5 risks type instability. Custom objectives require loss_function_expression. Learnable parameters use TemplateExpressionSpec(parameters=...); pre-1.4 function_symbols/lambda-style combine API is removed in 2.0.
TypeSpecTypeSpec generates Julia structs for nodes, independently of expression_spec, including fixed structures. Supply fields in declaration order: triple, then code; the nested tuple remains one field. Keep X 2D/y 1D and assign object cells individually to prevent extra NumPy axes:
import numpy as np
from pysr import PySRRegressor, TemplateExpressionSpec, TypeSpec
value = ((1.0, 2.0, 3.0), np.uint32(7))
X = np.empty((1, 1), dtype=object)
y = np.empty(1, dtype=object)
X[0, 0] = value
y[0] = value
packet = TypeSpec(
"Packet",
fields={"triple": "NTuple{3,Float64}", "code": "UInt32"},
sample="rng -> Packet(ntuple(_ -> randn(rng), 3), rand(rng, UInt32))",
mutate="""function (rng, value::Packet, temperature::Float64)
triple = ntuple(i -> value.triple[i] + temperature * randn(rng), 3)
bit = UInt32(1) << rand(rng, 0:31)
Packet(triple, xor(value.code, bit))
end""",
scalar_constants="value -> collect(value.triple)",
with_scalar_constants="(value, c) -> Packet((c[1], c[2], c[3]), value.code)",
is_valid="value -> all(isfinite, value.triple)",
)sample(rng) -> Packet: constant leaves; mutate(rng, value, temperature) -> Packet: continuous/discrete mutation; annotations are optional.scalar_constants(Packet) -> Vector{Float64} and with_scalar_constants(Packet, Vector{Float64}) -> Packet together; expose optimizable scalars, preserve other fields when rebuilding.is_valid(Packet) -> Bool: reject invalid intermediates; optional init: () -> Packet; string: Packet -> AbstractString.preamble: Julia before generated type/hooks; definitions: Julia immediately after type, including constructors/type-specific methods.loss_type: concrete AbstractFloat for full objectives; unavailable for elementwise losses, whose return type is inferred.
Configure PySRRegressor(type_spec=packet, operators=operators, elementwise_loss=loss). In arity-keyed operators, unary negate_packet(x::Packet)::Packet = Packet(ntuple(i -> -x.triple[i], 3), x.code) preserves code; binary combine_packets(x::Packet, y::Packet)::Packet = Packet(ntuple(i -> x.triple[i] + y.triple[i], 3), xor(x.code, y.code)) sums triples/XORs codes. Include binary choose_parameter(a::Packet, b::Packet) = a and choose_parameter(a::Packet, b::ValidVector) = ValidVector(map(_ -> a, b.x), b.valid). Define loss as packet_loss(p::Packet, y::Packet)::Float64 = sum((p.triple[i] - y.triple[i])^2 for i in 1:3) + Float64(count_ones(xor(p.code, y.code))); annotation keeps return type concrete.
Typed constructors work in ordinary_guess = ["combine_packets(x0, Packet((1.0, 2.0, 3.0), UInt32(7)))"] and template_component_guess = [{"f": "combine_packets(#1, Packet((1.0, 2.0, 3.0), UInt32(7)))"}]. Parsing uses the generated configuration module with Packet/operators defined; no parser closure/import needed.
Template parameters share the data's TypeSpec. Define packet_template = TemplateExpressionSpec(combine="choose_parameter(p[1], f(x))", expressions=["f"], variable_names=["x"], parameters={"p": 1}). Preserve one logical Packet with packet_guess_from_list = [{"f": "#1", "p": [value]}], or assign packet_parameter_array = np.empty(1, dtype=object) and packet_parameter_array[0] = value, then packet_guess_from_array = [{"f": "#1", "p": packet_parameter_array}]. Configure the estimator above with additional expression_spec=packet_template, guesses=packet_guess_from_array.
Verify field count/order, conversion to declared Julia types, type-stable Packet-returning operators, and one real scalar per loss pair; reserve Inf for invalid evaluations.Supply Julia strings and SymPy mappings using SymPy functions, never numpy/scipy; incorrect mappings break export/prediction:
model = PySRRegressor(
binary_operators=["+", "*"],
unary_operators=["inv(x) = 1/x", "gauss(x) = exp(-x^2)"],
extra_sympy_mappings={
"inv": lambda x: 1/x,
"gauss": lambda x: sympy.exp(-x**2),
},
)my_sqrt(x) = x >= 0 ? sqrt(x) : convert(typeof(x), NaN); any data-point NaN discards candidates with infinite loss. Prefer protected built-ins sqrt, log, acosh.T(2.5) with where {T}, or default-Float32 2.5f0; bare 2.5/0 introduce Float64/Int64 and break Float32 pipelines.operators or templates.from pysr import jl; jl.seval("import Pkg; Pkg.add(\"SpecialFunctions\")"); jl.seval("using SpecialFunctions"); safely wrap any importable function.class myop(sympy.Function): pass; symbolic export works, predict requires numeric mapping. Call model.refresh() after mapping changes.loss_functionUse full objectives for vectors, trees, derivatives, auxiliary data:
objective = """
function my_objective(tree, dataset::Dataset{T,L}, options) where {T,L}
prediction, completed = eval_tree_array(tree, dataset.X, options)
!completed && return L(Inf)
residuals = prediction .- dataset.y
return sum(abs2, residuals) / dataset.n
end
"""
model = PySRRegressor(loss_function=objective, binary_operators=["+", "*", "-"])completed before accessing predictions; failure leaves garbage. Return L(Inf) for numerical invalidity; use graded finite structural penalties such as L(1e6 * n_violations) so intermediate mutations survive. All-or-nothing structural rejection can make targets unreachable.dataset.X: features × samples, transposed from Python; dataset.y: vector; dataset.n: sample count. Three-argument objectives receive active batches; (tree, full_dataset, options, idx) exposes full data/selected indices.const globals or string interpolation, Distances.jl/standard packages; diagnose with @code_warntype/BenchmarkTools in separate Julia. Avoid internal-name collision eval_loss.eval_diff_tree_array for one feature, eval_grad_tree_array for all; support monotonicity/physics-informed losses. Templates use D(f, i)..sympy()/.predict(), requiring manual extraction.Before writing a traversal, check whether the rule is already an option. Operator placement rules are nested_constraints: "no sin inside cos and no cos inside sin" is nested_constraints={"sin": {"cos": 0}, "cos": {"sin": 0}}; "exponent must be a bare constant or variable" is constraints={"^": (-1, 1)} (keys are the names used in the operator lists, so "pow" if that is the name given). Both reject a violating mutation before it is evaluated, which is cheaper than a loss penalty and never lets a violator into the hall of fame. Use traversal only for what they cannot express: required features, constant value ranges, rules that depend on position or on several operators at once.
Implement those by traversal. Supported: any(f, tree), all(f, tree), count(f, tree), sum(f, tree), foreach(f, tree), collect(tree), depth-first for node in tree. Prefer functional traversal: loops allocate stacks; count/sum/any traverse directly.
node.degree: leaf 0/unary 1/binary 2; node.l, node.r: traversable children.node.constant distinguishes constants/variables; node.val: constant value; node.feature: 1-based feature index.node.op: 1-based supplied operator-list index, per arity.Allow ^ only with a lone constant exponent in [0,1]:
objective = """
function constrained_loss(tree, dataset::Dataset{T,L}, options) where {T,L}
idx_pow = 3 # position of ^ in binary_operators below (1-indexed)
n_bad = count(tree) do node
node.degree == 2 && node.op == idx_pow &&
any(c -> !(c.degree == 0 && c.constant && 0 <= c.val <= 1), node.r)
end
n_bad > 0 && return L(10_000 * n_bad)
prediction, valid = eval_tree_array(tree, dataset.X, options)
!valid && return L(Inf)
return sum(i -> abs2(prediction[i] - dataset.y[i]), eachindex(prediction)) / dataset.n
end
"""
model = PySRRegressor(binary_operators=["+", "*", "^"], loss_function=objective)Count violations before evaluation; multiply by a large finite penalty to guide evolution. Require feature 2: any(n -> n.degree == 0 && !n.constant && n.feature == 2, tree) || return L(big). Templates: obtain component trees using get_tree(ex) inside loss_function_expression. tree_mapreduce, NodeSampler, construction: https://ai.damtp.cam.ac.uk/dynamicexpressions/stable/examples/base_operations/
constraints={"pow": (9, 1)} caps argument complexity: base 9, lone constant/variable exponent; -1 unlimited. Strongly constrain ^; unrestricted exponentiation searches poorly.nested_constraints={"sin": {"sin": 0, "cos": 0}, "cos": {"sin": 0, "cos": 0}} prohibits nested trig; values cap inner occurrences within outer operators.complexity_of_operators={"exp": 3}, complexity_of_constants=2 discourage operators/constants; complexity_of_variables accepts global/per-feature costs.maxsize=35.maxsize counts operator/constant/variable nodes or complexity_mapping units; seven-feature linear models cost ~29, default 30 excludes many-term formulas.warmup_maxsize_by=0.5 ramps maxsize over that run fraction, helping avoid early complex traps.parsimony/adaptive_parsimony_scaling: defaults well-tuned in 1.5+.complexity_of_constants, or custom-operator rounding; continuous optimization does not guarantee integers.complexity_mappingReplace fixed operator prices/node counts with a Julia expression-to-Int function for emitted characters, evaluation cost, tokens, or depth. Pricing applies to front, maxsize, parsimony, constraints.
complexity_mapping = """
function source_chars(ex)
tree_mapreduce(
leaf -> leaf.constant ? length(string(leaf.val)) : 2, # "x0" is 2 chars
branch -> branch.degree == 1 ? 6 : 5, # "sin(" + ")" / " + " with parens
(parent, children...) -> parent + sum(children),
get_tree(ex),
)
end
"""
model = PySRRegressor(
binary_operators=["+", "-", "*"],
complexity_mapping=complexity_mapping,
maxsize=280,
)This mapping tracked every test-row len(equation) within offset 2 for outer parentheses; rank by actual cost and calibrate against emitted candidates.
AbstractExpression; get_tree(ex) exposes degree, constant, val, feature, op. tree_mapreduce/get_tree are in scope.length(string_tree(ex)): internal rendering uses binary_operator[3](...) and x1, x2 placeholders, unrelated to exported length.maxsize; default 30 with character pricing leaves equations_ empty when nothing fits. Hall of fame allocates one slot/unit through maxsize; filter long fine-grained fronts.guesses use identical pricing/acceptance rules.Check physical units during search:
model.fit(X, y, X_units=["Constants.M_sun", "kg", "m"], y_units="kg * m / s^2")"1" explicitly dimensionless.dimensional_constraint_penalty, default 1000; avoid 1e9, preserve graded paths through imperfect intermediates.[⋅]/[?] units; a lone constant can absorb any dimensions. Forbid with dimensionless_constants_only=True.parallelism="multithreading" suits laptops/single nodes. Thread count is fixed at Julia startup: set PYTHON_JULIACALL_THREADS=<n> before importing pysr; JULIA_NUM_THREADS is inappropriate under juliacall.populations ~2-3x threads/cores for worker utilization; default populations=31 covers typical machines.ncycles_per_iteration, reducing communication.parallelism="multiprocessing", procs=n costs substantially more startup/fit, potentially faster steady-state. Multiple nodes: cluster_manager="slurm" or native SlurmClusterManager.jl, https://ai.damtp.cam.ac.uk/symbolicregression/dev/slurm/. Launch once on one node, let it spawn workers, avoid srun; put worker packages in worker_imports.deterministic=True, random_state=<seed>, parallelism="serial"; parallel seeds remain nondeterministic. CPU differences can persist serially; reduce with precision=64.outputs/<run_id>/hall_of_fame.csv and checkpoint.pkl. Reload PySRRegressor.from_file(run_directory=...) with identical PySR version; pickles are version-locked. Preserve CSV/construction code durably; resupply custom extra_sympy_mappings.warm_start=True resumes previous populations on next same-process .fit(). Fix operators, maxsize, expression_spec, precision, feature count/order; loss/weights may change for staged objectives.model.sympy(i), model.latex(i), model.latex_table(), differentiable model.jax(i) returning {'callable', 'parameters'}, trainable model.pytorch(i). Supply custom extra_jax_mappings/extra_torch_mappings; fine-tune selected equations' constants by full-data gradient descent.precision=64 when important or values exceed ~1e19/fall below ~1e-19, risking Float32 overflow.| Symptom | Cause / fix |
|---|---|
| First import/fit takes minutes | Initial Julia download; per-process JIT. Retain process. |
| Compilation hangs | Apple Silicon: replace Rosetta x86 Python with native arm64. Notebook/embedded stdin wedge: input_stream="devnull". |
Startup lock.pid | Killed-process juliapkg lock: verify no installation, delete lock, import once freshly. Shared filesystems: pre-initialize before parallel jobs. |
Jupyter UnicodeDecodeError | Old PythonCall bug: upgrade or set PYTHON_JULIACALL_AUTOLOAD_IPYTHON_EXTENSION=no before import. |
| Cannot interrupt notebook | Known limitation; IPython/terminal q+Enter only. Set timeout_in_seconds or use IPython. |
| Nonsensical equations | Check scale/loss (log-space), missing/redundant operators, insufficient maxsize, excluding constraints, in order before more compute. |
| Tiny equations | Insufficient maxsize or MSE dominated by largest points. |
DomainError | Guard custom operators with typed NaN. |
| Prediction/SymPy fails | Correct extra_sympy_mappings; use SymPy functions. |
| Division near zero | Any-NaN-anywhere rejects candidates; survivors remain finite only on supplied data, potentially diverging elsewhere. |
Early-stop ProcessExitedException | Harmless multiprocessing worker-teardown noise. |
| Growing memory | Mostly fixed in recent Julia; upgrade or set multiprocessing heap_size_hint_in_bytes. |
| Different results | Stochastic evolution; apply serial determinism recipe. |
Use native SymbolicRegression.jl, SRRegressor via MLJ, for Julia pipelines/deep customization: expression types, mutation operators, per-component constraints. from pysr import jl exposes the runtime; jl.seval(...) executes arbitrary Julia, installed packages support operators/losses. Small readable backend extension points: src/Options.jl, src/CheckConstraints.jl; connect dev checkouts through pysr/juliapkg.json.
Target: PySR 2.0.0, SymbolicRegression.jl 2.x. Relative to 1.5.x: n-ary operators={1: ["sin"], 2: ["+", "*"], 3: ["clamp"]}, guesses, TypeSpec, automatic batching, autodiff plugins. Defaults: batching="auto", batch_size=None, annealing=True, crossover_probability=0.2, new mutation mix; use 2.0 defaults rather than 1.x tuning.
| Deprecated | Current |
|---|---|
pysr.install(), python -m pysr install, PyCall/from julia import Main | Automatic Julia installation on import |
pysr(...), best(), get_hof() | PySRRegressor, .fit(), model.equations_, get_best() |
multithreading=True/False, procs=0 | parallelism="multithreading"/"multiprocessing"/"serial" |
full_objective= | loss_function; templates: loss_function_expression |
loss= | elementwise_loss |
equation_file= | output_directory= + run_id= |
npop, ncyclesperiteration, camelCase args | population_size, ncycles_per_iteration, snake_case |
Positional TemplateExpressionSpec, function_symbols=... | Explicit combine=, expressions=, variable_names= |
ad266a8
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.