add ultranest import #313

JohannesBuchner · 2023-07-11T07:24:46Z

Description

Fixes #311

adds

a test data set for ultranest outputs
a script that created it
a test that verifies a ultranest reader
a ultranest output reader
anesthetic GUI support for ultranest

Checklist:

I have performed a self-review of my own code
My code is PEP8 compliant (flake8 anesthetic tests)
My code contains compliant docstrings (pydocstyle --convention=numpy anesthetic)
New and existing unit tests pass locally with my changes (python -m pytest)
I have added tests that prove my fix is effective or that my feature works
I have appropriately incremented the semantic version number in both README.rst and anesthetic/_version.py

…ix style

a bit worrisome that this did not give an error before

lukashergt

Hi @JohannesBuchner,

thanks for setting this up!

In the interest of keeping things slim, we try to only keep the minimum needed amount of example_data files that are needed for anesthetic to produce its dataframes.

So e.g. in the case of PolyChord which produces all kinds of output files, we only use the following three files:

<root>_dead-birth.txt
<root>.paramnames
<root>_phys_live-birth.txt

JohannesBuchner · 2023-07-11T10:42:31Z

OK, done

JohannesBuchner · 2023-07-11T12:02:22Z

The h5py import is failing due to no dependency on it. If you want to keep it optional, we can move it into the read_ultranest function. commit caca2f2 does that.

…ches between key and value

codecov · 2023-07-11T15:54:33Z

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (cb2759c) 100.00% compared to head (76c982f) 100.00%.

Additional details and impacted files

@@            Coverage Diff            @@
##            master      #313   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           32        33    +1     
  Lines         2764      2790   +26     
=========================================
+ Hits          2764      2790   +26

Files Changed	Coverage Δ
anesthetic/read/cobaya.py	`100.00% <ø> (ø)`
anesthetic/read/multinest.py	`100.00% <ø> (ø)`
anesthetic/read/polychord.py	`100.00% <ø> (ø)`
anesthetic/_version.py	`100.00% <100.00%> (ø)`
anesthetic/read/chain.py	`100.00% <100.00%> (ø)`
anesthetic/read/ultranest.py	`100.00% <100.00%> (ø)`
anesthetic/samples.py	`100.00% <100.00%> (ø)`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

… `un`

tests/example_data/makeun.py

williamjameshandley

Looking good. Many thanks @JohannesBuchner and @lukashergt. Only minor comments/adjustments/organisation to be addressed before merging.

This PR should remind us that it really would be helpful to finish #158 for dynesty support; although until dynesty returns birth contours we'll need to allow constructors to provide a dynamic nlive in the same way that at the moment you can provide fixed nlive in place of logL_birth.

anesthetic/plot.py

anesthetic/read/ultranest.py

tests/example_data/un/info/results.json

williamjameshandley · 2023-07-19T09:58:26Z

tests/example_data/un/info/results.json

If I understand correctly, the only thing we need from this is the paramnames entry, although it is nice that it has a maximum likelihood point. Presuming this is found by an optimisation procedure (like e.g. polychord's polishing settings.maximise=true gives) then this is could be useful in future iterations.

This is the last live point discarded, but for the default frac_remain=0.01 this is good enough.

Theoretically I disagree -- the rule of thumb is that logLmax ~ <logL>_P + d/2, and the width of the typical set in logL space is sqrt(d/2), so in even moderate dimensions the true likelihood peak lies some distance away from where nested sampling terminates, regardless of stopping criterion (see figure 2), but this isn't relevant to this PR!

I have been wondering if we could somehow make use of PolyChord's maximise=true output in anesthetic...

What happens when one appends the list of nested samples with the maximum likelihood point at the very end?

For safety we could do it with zero weight...?

Or is there a way of turning the maximum likelihood point into the final nested sampling point?

I think in practice all of these would work fine for a run that had reached convergence. I would worry if you also started trying to use it at different temperatures.

As an API, a better solution would be for the maximum to be appended with a logL_birth=logL=logL_max, so it's officially 'beyond the last live point'.

We'd then have to write the volume calculation to ensure anesthetic set these to zero or nan weight by default, since there's no way to determine the volume if there's a gap.

I could imagine gaps occurring naturally in the case of importance weighting.

Issue raised in #317.

Theoretically I disagree -- the rule of thumb is that logLmax ~ <logL>_P + d/2, and the width of the typical set in logL space is sqrt(d/2), so in even moderate dimensions the true likelihood peak lies some distance away from where nested sampling terminates, regardless of stopping criterion (see figure 2), but this isn't relevant to this PR!

As an aside, I have been working through this analytically and experimentally this afternoon, and this statement only holds in higher dimensions.

Here is an analytic result for f=0.01, n=1000 for how close you get in loglikelihood to the maximum as a function of dimension:

tests/test_gui.py

docs/source/reading_writing.rst

tests/example_data/un_gen_data.py

JohannesBuchner · 2023-07-19T11:49:04Z

I won't have time to look at this before my visit to Cambridge in September, lots of travel and conferences at the moment.

…DataFrame._set_labels` is a dict, it should be expected to be a dict, and if it is not, then it should fail and not silently ignore `labels`

…ls()` as `make_2d_axes` expects a dict and not a list

…tially assigned labels

…o MultiNest files

…ta generating code next to its output

williamjameshandley

Nice. I'm happy for this to be merged -- not sure who wants to claim the github credits -- it was initiated by @JohannesBuchner (who may be too out-of-action for the next 6 weeks to press the button), but @lukashergt has probably by this point adjusted more lines.

williamjameshandley · 2023-07-19T21:01:30Z

Whoever does it should press 'squash and merge'.

JohannesBuchner · 2023-07-19T21:24:48Z

I don't have the commit bit on this repo ...

…correctly

williamjameshandley · 2023-07-19T22:32:19Z

I don't have the commit bit on this repo ...

I've just sent you a new invitation which should allow you to.

lukashergt · 2023-07-19T22:56:15Z

Nice. I'm happy for this to be merged -- not sure who wants to claim the github credits -- it was initiated by @JohannesBuchner (who may be too out-of-action for the next 6 weeks to press the button), but @lukashergt has probably by this point adjusted more lines.

Whoever does it should press 'squash and merge'.

I don't have the commit bit on this repo ...

I've just sent you a new invitation which should allow you to.

It should ideally be @JohannesBuchner to squash and merge (which should hopefully be possible now), but also ideally sooner than in 6 weeks time. However, no obligation if things are too busy, then I'll do it in a couple of days.

JohannesBuchner · 2023-07-19T23:38:50Z

I pressed buttons ¯\(ツ)/¯

lukashergt · 2023-07-19T23:41:49Z

Great! Thanks @JohannesBuchner, and thanks to MaxEnt 2023, which made this happen.

JohannesBuchner added 3 commits July 11, 2023 08:52

add ultranest import

a6286cd

add test and output data from ultranest and script that creates it; f…

592eefe

…ix style

typo

4877bd7

a bit worrisome that this did not give an error before

lukashergt reviewed Jul 11, 2023

View reviewed changes

minimize ultranest data included

fcabe8e

move h5py import into function, to make it only run when needed

caca2f2

lukashergt added 7 commits July 11, 2023 15:15

version bump to 2.1.0

6283f88

make ultranest part of gui tests

bb1eebd

fix test_gui_params to work for both pc and un

b8cd8ab

put json and h5py into optional dependencies

adb1ebc

remove json from optional dependencies as it is a built-in package

8d02f60

fix labelling of AxesDataFrame for the case where there are exact mat…

070b75e

…ches between key and value

skip tests related to ultranest if h5py not in sys.modules

7033eab

lukashergt added 3 commits July 11, 2023 16:59

remove trailing print statement

441c1d7

update docs for reading UltraNest samples

639bb09

rename makeun.py to un_gen_data.py such that it gets grouped with…

ce41b63

… `un`

lukashergt reviewed Jul 11, 2023

View reviewed changes

tests/example_data/makeun.py Outdated Show resolved Hide resolved

add Johannes Buchner to list of contributors

58baf2c

lukashergt requested a review from williamjameshandley July 11, 2023 16:45

lukashergt mentioned this pull request Jul 17, 2023

Add specific docs for the GUI #315

Merged

7 tasks

williamjameshandley requested changes Jul 19, 2023

View reviewed changes

lukashergt added 5 commits July 19, 2023 14:13

revert commit 070b75e, which checked whether labels passed to `Axes…

8b09789

…DataFrame._set_labels` is a dict, it should be expected to be a dict, and if it is not, then it should fail and not silently ignore `labels`

pass labels=self.get_labels_map() rather than `labels=self.get_labe…

8a060fd

…ls()` as `make_2d_axes` expects a dict and not a list

add test for axes labels when dropping labels or when only having par…

63e264f

…tially assigned labels

correct read_ultranest docstring to refer to UltraNest files, not t…

c9891f9

…o MultiNest files

add UltraNest to the list of supported file structures in read_chains

b894efb

lukashergt added 6 commits July 19, 2023 15:12

improve read_cobaya, read_multinest and read_polychord docstrings

f6c62cf

move bin/gen_data.py to tests/example_data/gen_data.py to have da…

298d8d9

…ta generating code next to its output

bring gen_data.py to the anesthetic 2.0.0 century

64e7349

make gen_data.py flake8 conform

1acbf53

temporarily remove un/info/results.json

71c8cf0

add un/info/results.json back in, hopefully in the right format

0ab5210

lukashergt mentioned this pull request Jul 19, 2023

Make sampling code from gen_data.py accessible in anesthetic.examples #316

Open

add mn to the gui tests

d6da757

lukashergt mentioned this pull request Jul 19, 2023

integrate maximum likelihood point capabilities #317

Open

lukashergt requested a review from williamjameshandley July 19, 2023 18:15

williamjameshandley previously approved these changes Jul 19, 2023

View reviewed changes

Merge branch 'master' into master

57b00e1

williamjameshandley dismissed their stale review via 57b00e1 July 19, 2023 22:26

Added back in infuriating newline which github automerge fails to do …

76c982f

…correctly

williamjameshandley approved these changes Jul 19, 2023

View reviewed changes

JohannesBuchner merged commit d3aafc2 into handley-lab:master Jul 19, 2023

williamjameshandley mentioned this pull request Jul 25, 2023

[Draft] Support for converting from dynesty and emcee samples #158

Draft

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add ultranest import #313

add ultranest import #313

JohannesBuchner commented Jul 11, 2023 •

edited by lukashergt

Loading

lukashergt left a comment

JohannesBuchner commented Jul 11, 2023

JohannesBuchner commented Jul 11, 2023 •

edited

Loading

codecov bot commented Jul 11, 2023 •

edited

Loading

williamjameshandley left a comment •

edited

Loading

williamjameshandley Jul 19, 2023

JohannesBuchner Jul 19, 2023 •

edited

Loading

williamjameshandley Jul 19, 2023 •

edited

Loading

lukashergt Jul 19, 2023 •

edited

Loading

williamjameshandley Jul 19, 2023

lukashergt Jul 19, 2023

williamjameshandley Jul 19, 2023

JohannesBuchner commented Jul 19, 2023

williamjameshandley left a comment

williamjameshandley commented Jul 19, 2023

JohannesBuchner commented Jul 19, 2023

williamjameshandley commented Jul 19, 2023

lukashergt commented Jul 19, 2023 •

edited

Loading

JohannesBuchner commented Jul 19, 2023 •

edited

Loading

lukashergt commented Jul 19, 2023

add ultranest import #313

add ultranest import #313

Conversation

JohannesBuchner commented Jul 11, 2023 • edited by lukashergt Loading

Description

Checklist:

lukashergt left a comment

Choose a reason for hiding this comment

JohannesBuchner commented Jul 11, 2023

JohannesBuchner commented Jul 11, 2023 • edited Loading

codecov bot commented Jul 11, 2023 • edited Loading

Codecov Report

williamjameshandley left a comment • edited Loading

Choose a reason for hiding this comment

williamjameshandley Jul 19, 2023

Choose a reason for hiding this comment

JohannesBuchner Jul 19, 2023 • edited Loading

Choose a reason for hiding this comment

williamjameshandley Jul 19, 2023 • edited Loading

Choose a reason for hiding this comment

lukashergt Jul 19, 2023 • edited Loading

Choose a reason for hiding this comment

williamjameshandley Jul 19, 2023

Choose a reason for hiding this comment

lukashergt Jul 19, 2023

Choose a reason for hiding this comment

williamjameshandley Jul 19, 2023

Choose a reason for hiding this comment

JohannesBuchner commented Jul 19, 2023

williamjameshandley left a comment

Choose a reason for hiding this comment

williamjameshandley commented Jul 19, 2023

JohannesBuchner commented Jul 19, 2023

williamjameshandley commented Jul 19, 2023

lukashergt commented Jul 19, 2023 • edited Loading

JohannesBuchner commented Jul 19, 2023 • edited Loading

lukashergt commented Jul 19, 2023

JohannesBuchner commented Jul 11, 2023 •

edited by lukashergt

Loading

JohannesBuchner commented Jul 11, 2023 •

edited

Loading

codecov bot commented Jul 11, 2023 •

edited

Loading

williamjameshandley left a comment •

edited

Loading

JohannesBuchner Jul 19, 2023 •

edited

Loading

williamjameshandley Jul 19, 2023 •

edited

Loading

lukashergt Jul 19, 2023 •

edited

Loading

lukashergt commented Jul 19, 2023 •

edited

Loading

JohannesBuchner commented Jul 19, 2023 •

edited

Loading