Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize and add documentation #543

Merged
merged 8 commits into from
Oct 8, 2024
Merged

Reorganize and add documentation #543

merged 8 commits into from
Oct 8, 2024

Conversation

ztangent
Copy link
Member

@ztangent ztangent commented Oct 5, 2024

Motivation

After using the refactored documentation introduced in #526, I found it harder to navigate the Gen.jl docs than before, in part because reference material (e.g. explanations of the semantics of the GFI, MCMC kernels, and trace translators) was migrated to tutorials and how-tos.

I understand that some of these changes were made in order to follow the documentation system described here, but I actually think the strict separation between explanation and reference material recommended by that system is not the best fit for Gen. For key Gen features like the GFI and the Trace Translator system, as well as the numerous DSLs (e.g. the @dist DSL, or the compositional kernel DSL), reference material needs to explain the semantics of the language, and IMO that is best done through prose and examples.

As such, I think the docs become less usable if we restrict the reference section to just describing function APIs, instead of also explaining the underlying concepts. Perhaps there is some version of the Gen documentation that would cleanly separate explanation from the API, but I think we're not ready to move to that yet without substantial rewriting of the material in the docs.

In any case, I should have reviewed the documentation changes in #526 more closely before releasing the most recent version of Gen --- apologies!

Changes in this PR

All that said, I think the Gen.jl docs were long due for an overhaul, and I'm glad #526 got that started! This PR reorganizes the documentation to address the issues described above, resulting in a documentation structure as follows:

  • Gen.jl: Home page, with a list of features, installation instructions, and links to the Tutorials (including Getting Started)
  • Tutorials:
    • Getting Started
    • Introduction to Modeling in Gen
    • Learning Generative Functions (previously this was in Reference / API, but it really was a tutorial)
    • etc.
  • How-to-Guides
    • Mostly stuff on how to extend and customize Gen at this point
  • Reference
    • Core Interfaces (GFI, Choice Maps, Selections, Change Hints)
    • Modeling Library (Modeling Languages, Combinators, Custom Gen Functions etc.)
    • Inference Library (Trace Translators and Parameter Optimization moved here from Modeling Library)
    • Internals (Currently contains only a rough description of the modeling language implementation)

This PR also contributes a number of additional changes:

  • More documentation of Particle Filtering / SMC, with references to GenParticleFilters.jl and SMCP3
  • More documentation of MAP Optimization
  • Added documentation for Wake-Sleep Learning
  • Added/completed a full tutorial for Variational Inference in Gen (including amortized VI)
  • Minor fixes to accumulate_param_gradients! for Map and Unfold, required for the above to work.

Finally, this PR enables documentation previews for new pull requests, and a new GitHub action that cleans up the doc site previews once the PR is merged. This should make it much easier to review changes to the documentation going forward (without having to build locally).

The documentation preview for this PR should be available here once it is deployed: https://www.gen.dev/docs/previews/PR543

@limarta
Copy link
Contributor

limarta commented Oct 7, 2024

This is great. Fwiw, I also tried emulating the tutorial structure based off these SciML packages.

@ztangent
Copy link
Member Author

ztangent commented Oct 8, 2024

Thanks for taking a look at this! Will merge now.

@ztangent ztangent merged commit a5fc8e3 into master Oct 8, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants