RED - Residuals for Equation Discovery

Residuals for Equation Discovery (RED) is a post-processing method for Equation Discovery that improves a given equation in a targeted manner, based on its residuals

We test the method on five equation discovery systems (EDS) NeSymReS, SymbolicGPT, E2E, PySR and GPLearn.

In total, we implemented six methods to postprocess the initial equation suggested by the EDS (RED, Permute, Hyper Parameter Grid, Constant Fitting, CVGP, Seeded GPlearn)

This repository Includes the code for the paper Brugger, J., Pfanschilling, V., Richter, D., Mezini, M., Kramer, S.: Prompting neural- guided equation discovery based on residuals. In: Discovery Science (2025) https://doi.org/10.1007/978-3-032-05461-6_7 a preprint is on arxiv
https://doi.org/10.48550/arXiv.2511.05586

Install RED

We represent equations as syntax trees. The code for this is integrated via a git submodule. It behaves like a separate repro in the main project. In the best case, the submodule should be downloaded with the following command from the project root

git submodule update --init --recursive

If it doesn't work, we can also download it manually.

 git submodule add https://github.com/wwjbrugger/SyntaxTree.git SyntaxTree

# Prompting Neural-Guided Equation Discovery Based on Residuals
git clone https://github.com/SymposiumOrganization/NeuralSymbolicRegressionThatScales.git
cd NeuralSymbolicRegressionThatScales/
pip3 install -e src/

All packages are specified for Python version 3.10

pip install "cython<3.0.0" wheel
pip install "pyyaml==5.4.1" --no-build-isolation
pip install -r requirements.txt
python -m pip install numpy==1.26.4

Experiments

The experiments in section 4.1 - 4.4 can be reproduced using the commands specified in experiments.sh, revision 'f7e92c79eacd33d6b13c6a325ffc08e87bca6b07 (Camera Ready Discovery Science v_0)' and analyse_experiments/compare_results_acc.py . Experiments in section 4.5 can be reproduced using the commands in experiments_pool_nmse.sh, revision 04763994e474763d2525f074072e175a5bd7578b and analyse_experiments/ablation_pool.py

The scripts src/fit_func_[model].py start the experiments for the respective model. The script src/experiment_schedule.py is the backbone which orchestrate the experiments. The postprocessing methods are located in src/post_processing_methods.

Add new Equation Discovery Model

We only need a method to load the pretrained model and a method which handels the call of the model.

def load_model():
    model = ...
    fitfunc = partial(call,
                      model=model,
                      args=args
                      )
    return fitfunc


def call(model, X_df, Y_df, args):
    equation = model.fit(X_df.to_numpy(), Y_df.to_numpy())
    equation = str(sympify(equation))
    output = evaluate_equation(args, equation, X_df, Y_df)
    return output

Add a new Operator

New Operator can be added in src/equation_class/math_class.
Afterward, they have to be added in src/syntax_tree/syntax_tree.py in the dictionary self.operator_to_class and in src/equation_classes/infix_to_prefix.py in the method getPriority