Update to the AdvancedVI@0.4 interface by Red-Portal · Pull Request #2506 · TuringLang/Turing.jl

Red-Portal · 2025-03-14T23:14:25Z

This PR aims to update Turing's Variational module to match AdvancedVI's new interface starting from v0.3. I will try not to change the interface too much, but given the new features in AdvancedVI, I think breaking changes will be inevitable. Though the focus will be to provide a good default setting rather than to expose all the features.

Currently proposed interface:

using Turing
using AdvancedVI

d = randn(100)
Turing.@model function model()
   x ~ MvNormal(randn(100), 1)
   y ~ InverseGamma()
end
m = model()
q = Turing.Variational.q_fullrank_gaussian(Random.default_rng(), m)
n_iters = 1000
Turing.Variational.vi(m, q, n_iters)

Closes #2507
Closes #2508
Closes #2430

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…nto update_advancedvi

src/variational/VariationalInference.jl

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…nto update_advancedvi

yebai · 2025-03-20T21:21:36Z

@Red-Portal, can you fix the tests before I take a look?

Red-Portal · 2025-03-20T21:31:09Z

@yebai I marked the PR as a draft so that we can first agree on an interface, and then I flesh out the implementation and the tests. Do we wish we proceed in another way?

yebai · 2025-03-20T21:51:41Z

Let's address the interface later or in a separate PR since that might require more discussions. For this PR, let's try to keep the VI interface non-breaking where possible.

yebai

Many thanks, @Red-Portal, for the excellent work!

I only have one comment on the interface. See below.

yebai · 2025-05-29T21:37:36Z

src/variational/VariationalInference.jl

+- `state`: Collection of states used for optimization. This can be used to resume from a past call to `vi`.
+- `info`: Information generated during the optimization run.
+"""
+function vi(


This function is a thin wrapper around AdvancedVI.optimize. I'd suggest we consider #2509 (comment) for this interface to make it future-proof:

optimise(model, VI(q, n_iterations, objective; ...), ...)

Given that the function vi itself is more of a legacy from the v0.2 days, I suggest we first go with this and add the new interface in the future? Whatever we decide to do, I think having vi now won't be too much of a hassle in the future. (On that matter, I am sympathetic to the proposal to unify everything into infer)

Co-authored-by: Penelope Yong <penelopeysm@gmail.com>

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

penelopeysm

Just the changelog entry, please!

Red-Portal · 2025-06-02T20:17:36Z

Ah crap forgot about that one. Sorry will do.

penelopeysm

Excellent work 😄

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

yebai · 2025-06-03T16:19:56Z

Thanks @Red-Portal!

Red-Portal · 2025-06-03T21:35:16Z

@yebai @penelopeysm Thank you both for all the feedback!

@yebai

* [skip ci] Bump minor version * Clean up old code (#2574) * Remove src/essential and deprecated function stubs * Fix imports * Export `@addlogprob!` * Fix more tests * Clean up more stuff * Update to the AdvancedVI@0.4 interface (#2506) * update to match the AdvancedVI@0.3 interface * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * remove plotting * fix formatting Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix formatting Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * fix formatting Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * remove unused dependency * Update Project.toml * fix make some arugments of vi initializer to be optional kwargs * remove tests for custom optimizers * remove unused file * Update src/variational/bijectors.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Update Turing.jl * fix remove call to `AdvancedVI.turnprogress`, which has been removed * apply comments from @yebai * Update src/variational/VariationalInference.jl Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add old interface as deprecated * bump AdvancedVI version * add deprecation for `meanfield` * add `default_rng` interfaces * add tests for variational inference * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * remove "src/variational/bijectors.jl" (moved to `DynamicPPL.jl`) * add more tests for variational inference initializer * remove non-essential reexports, fix tests * run formatter, rename functions * add documentation * fix run formatter * fix remove debug commits * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * run formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * add Variational submodule * fix docstring style * update docstring style * format docstring style * fix typo Co-authored-by: Penelope Yong <penelopeysm@gmail.com> * fix use fixed seed with StableRNGs * fix export variational families * fix forma Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * update changelog for advancedvi 0.4 * fix version number * Format & add some links * fix formatting Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Hong Ge <3279477+yebai@users.noreply.github.com> Co-authored-by: Penelope Yong <penelopeysm@gmail.com> * [no ci] fix changelog --------- Co-authored-by: Kyurae Kim <msca8h@naver.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Hong Ge <3279477+yebai@users.noreply.github.com>

Red-Portal and others added 12 commits March 14, 2025 19:09

update to match the AdvancedVI@0.3 interface

ed6946c

run formatter

a94269d

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

a4711a9

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

3f8068b

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

222a638

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

57097f5

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

a42eea8

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

798f319

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

69a4972

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

run formatter

cbcb8b5

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

remove plotting

081d6ff

Merge branch 'update_advancedvi' of github.com:TuringLang/Turing.jl i…

a32a673

…nto update_advancedvi

Red-Portal marked this pull request as draft March 14, 2025 23:23

github-actions bot reviewed Mar 14, 2025

View reviewed changes

src/variational/VariationalInference.jl Outdated Show resolved Hide resolved

Red-Portal and others added 5 commits March 14, 2025 19:29

fix formatting

1bcec3e

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix formatting

b142832

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

fix formatting

061ec35

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

remove unused dependency

736bd3e

Merge branch 'update_advancedvi' of github.com:TuringLang/Turing.jl i…

fd434d8

…nto update_advancedvi

Red-Portal mentioned this pull request Mar 16, 2025

AdvancedVI 0.3 and compatibility with Turing.jl #2430

Closed

Red-Portal requested review from mhauru, torfjelde and yebai March 16, 2025 14:08

yebai mentioned this pull request Mar 16, 2025

Interface for MAP/MLE/VI #2509

Open

Merge branch 'main' into update_advancedvi

57108ee

yebai added 2 commits March 20, 2025 21:21

Merge branch 'main' into update_advancedvi

8dc8067

Update Project.toml

297c32a

penelopeysm force-pushed the breaking branch 2 times, most recently from 16c62f2 to c2ae953 Compare May 28, 2025 20:49

Merge branch 'breaking' into update_advancedvi

ea417fc

yebai requested changes May 29, 2025

View reviewed changes

Red-Portal and others added 3 commits May 30, 2025 17:10

fix typo

4c9a538

Co-authored-by: Penelope Yong <penelopeysm@gmail.com>

fix use fixed seed with StableRNGs

dfa8d20

fix export variational families

a18f581

Red-Portal requested review from penelopeysm and yebai May 30, 2025 22:47

fix forma

f9528e0

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

yebai approved these changes May 30, 2025

View reviewed changes

yebai changed the base branch from breaking to main May 30, 2025 22:58

Merge branch 'main' into update_advancedvi

fb150c7

yebai changed the base branch from main to breaking May 30, 2025 22:59

Merge remote-tracking branch 'origin/breaking' into update_advancedvi

8174725

penelopeysm requested changes Jun 2, 2025

View reviewed changes

update changelog for advancedvi 0.4

dec108b

Red-Portal requested a review from penelopeysm June 2, 2025 22:43

Red-Portal and others added 2 commits June 2, 2025 18:44

fix version number

b0d791e

Format & add some links

29373ee

penelopeysm approved these changes Jun 2, 2025

View reviewed changes

Red-Portal and others added 2 commits June 3, 2025 12:22

Merge branch 'breaking' into update_advancedvi

d21e652

fix formatting

4c72501

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

yebai merged commit ba10394 into breaking Jun 3, 2025
31 of 33 checks passed

yebai deleted the update_advancedvi branch June 3, 2025 16:19

yebai mentioned this pull request Jun 3, 2025

Turing 1.0 Meta Meta Issue #2420

Open

18 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to the AdvancedVI@0.4 interface#2506

Update to the AdvancedVI@0.4 interface#2506
yebai merged 69 commits intobreakingfrom
update_advancedvi

Red-Portal commented Mar 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

yebai commented Mar 20, 2025

Uh oh!

Red-Portal commented Mar 20, 2025

Uh oh!

yebai commented Mar 20, 2025

Uh oh!

yebai left a comment

Uh oh!

yebai May 29, 2025

Uh oh!

Red-Portal May 30, 2025

Uh oh!

penelopeysm left a comment

Uh oh!

Red-Portal commented Jun 2, 2025

Uh oh!

penelopeysm left a comment

Uh oh!

Uh oh!

yebai commented Jun 3, 2025

Uh oh!

Red-Portal commented Jun 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

Red-Portal commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

yebai commented Mar 20, 2025

Uh oh!

Red-Portal commented Mar 20, 2025

Uh oh!

yebai commented Mar 20, 2025

Uh oh!

yebai left a comment

Choose a reason for hiding this comment

Uh oh!

yebai May 29, 2025

Choose a reason for hiding this comment

Uh oh!

Red-Portal May 30, 2025

Choose a reason for hiding this comment

Uh oh!

penelopeysm left a comment

Choose a reason for hiding this comment

Uh oh!

Red-Portal commented Jun 2, 2025

Uh oh!

penelopeysm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yebai commented Jun 3, 2025

Uh oh!

Red-Portal commented Jun 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Red-Portal commented Mar 14, 2025 •

edited

Loading