Add callbacks to all solver states and interfaces, and start a developer guide section

[kellertuer]

This PR introduces callbacks into the solver interface by adding a field to all solvers.
The documentation is refactored to some extend to split the formerplan into a Developer guid (base/) and the library section of common elements that solvers share (commons) – which is also reflected in the code organisation.

Solvers to refactor

• adaptive_regularization_with_cubics (AdaptiveRegularizationState)
• alternating_gradient_descent (AlternatingGradientDescentState)
• augmented_Lagrangian_method (AugmentedLagrangianMethodState)
• convex_bundle_method (ConvexBundleMethodState)
• ChambollePock (ChambollePockState)
• cma_es (CMAESState) - state is now also exported
• conjugate_gradient_descent (ConjugateGradientDescentState)
• conjugate_residual (ConjugateResidualState)
• CoordinatesNormalSystemState not necessary
• cyclic_proximal_point (CyclicProximalPointState)
• difference_of_convex_algorithm (DifferenceOfConvexState)
• difference_of_convex_proximal_point (DifferenceOfConvexProximalState)
• DouglasRachford (DouglasRachfordState)
• exact_penalty_method (ExactPenaltyMethodState)
• Frank_Wolfe_method (FrankWolfeState)
• gradient_descent (GradientDescentState)
• gradient_sampling (GradientSamplingState) state is now also exported
• interior_point_Newton (InteriorPointNewtonState)
• LanczosState
• LevenbergMarquardt (LevenbergMarquardtState)
• LevenbergMarquardtBoxSubsolver not necessary
• mesh_adaptive_direct_search (MeshAdaptiveDirectSearchState)
• NelderMead (NelderMeadState)
• particle_swarm (ParticleSwarmState)
• primal_dual_semismooth_Newton (PrimalDualSemismoothNewtonState)
• projected_gradient_method (ProjectedGradientMethodState)
• proximal_bundle_method (ProximalBundleMethodState)
• proximal_gradient_method (ProximalGradientMethodState)
• proximal_point (ProximalPointState)
• quasi_Newton (QuasiNewtonState)
• StepsizeState not necessary, mainly serves as an interims storage.
• stochastic_gradient_descent (StochasticGradientDescentState)
• subgradient_method (SubGradientMethodState)
• truncated_conjugate_gradient_descent (TruncatedConjugateGradientState)
• trust_regions (TrustRegionsState)
• vectorbundle_newton (VectorBundleNewtonState)

In the end

• remove the warning in get_callbacks once all states have that implemented, cf.

  Manopt.jl/src/base/state/callback.jl

  Lines 48 to 52 in 15e7e54

  	@warn """
  	This is a safety fallback! Upon initialization/setup, reaching this means your callback(s)
  	are not stored in the state of type $(typeof(state).name).
  	Reaching this during a solver run, means your callbacks will not be called
  	"""

  – then this PR is even nonbreaking, since this accessor has a “default empty return”

[kellertuer]

If this is not so urgent, then I can continue a bit refactoring the docs; I can also do that on a next PR though.

Besides looking at the changes once more, we should also properly deprecate the callback= keyword in the decoration of a state (and test the warning instead of showing one in the tests) – and deprecate the DebugCallback action.

[mateuszbaran]

I've added a small tutorial for callbacks, fixed coverage regression, and fixed one bug I've encountered while fixing the regression (HZ status summary not working right). provided_callbacks maybe should be exported now?

[kellertuer]

I think we can postpone further rework on the docs a bit and merge this.

Reworking the docs will also require a bit of moving function definitions around – I want to get rid of the plan/ folder basically, but that should maybe not delay this PR.

[mateuszbaran]

Sure, I think this looks good now (up to that one character 🙂 )

[kellertuer]

Nice. I am just leaving right now, but I could bump version and such tomorrow morning and then merge and register.