Add callback-related tests

[chrhansk]

No description provided.

[chrhansk]

@TimSiebert1: Is it possible to run unit tests in pull requests as well?

[TimSiebert1]

Hi @chrhansk,

Thanks for adding tests! Unfortunately, the tests have to be added into the folder boost-test and to this CmakeLists. Then the tests will run in the workflow for PRs (see here).

I saw that you are using headers that I removed on master while refactoring. You can rely on the main header adolc/adolc.h, which would be the recommended way to include adolc.

If you have any questions, don't hesitate to ask for support. :)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 40.62%. Comparing base (c104adb) to head (3cad81b).
⚠️ Report is 9 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #157      +/-   ##
==========================================
+ Coverage   40.61%   40.62%   +0.01%     
==========================================
  Files          52       52              
  Lines       14256    14254       -2     
==========================================
+ Hits         5790     5791       +1     
+ Misses       8466     8463       -3     

Flag	Coverage Δ
unittests	40.62% <ø> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[chrhansk]

@TimSiebert1:

I noticed that to evaluate Hessians, the callback function hos_ti_reverse is being called by ADOL-C.

Unfortunately, this function is not well documented despite being used in the interface. I can see here

https://github.com/coin-or/ADOL-C/blob/master/ADOL-C/src/ho_rev.cpp#L223-L239

that hos_reverse is implemented using this function internally. Essentially, hos_ti_reverse has (d+1)xm Lagrange multipliers, but when used to implement hos_reverse on the 0xm ones are set to a non-zero value. Could you tell me what the purpose of the other multipliers is?

[TimSiebert1]

The method hos_ti_reverse stands for "higher-order-scalar-taylor-input-reverse" if I'm correctly. This doesn't help that much, unless you know that "taylor-input" means you are allowed to specify a distinct weight for every Taylor coefficient for the reverse mode. So instead of getting w^T*F_0, w^T*F_1, ..., w^T*F_d as you get for hos_reverse , you get w_0^T * F_0, ..., w_d^T * F_d.

Despite being used internally for the higher-order methods, I'm not sure if there ever was a use case for having such freedom.

Hope that has made things a little clearer.

[chrhansk]

Hmm, my problem is the following: hos_reverse should compute

$$ \left[ w F'(x_0), w F''(x_0)x_1, w F'''(x_0) x_1 x_2 , \ldots \right], $$

i.e., it should use the same (and generally non-zero) Lagrange multiplier for all degrees from 1 to d + 1.

If hos_ti_reverse would work as you say, than I would implement hos_reverse by setting

$$ w_0 \equiv w_1 \equiv w_{d} \equiv w $$

rather than

$$ w_0 = w, w_{k} = 0 \text{ for } k = 1, \ldots, d $$

or am I reading something wrong?

[TimSiebert1]

Sorry, I didn't write it down in a nice way. The method hos_reverse computes the derivatives of the Taylor polynomial it gets from fos_forward or hos_forward. Assume we get from the forward pass $[y_0, y_1, ..., y_d]$ denoting the corresponding Taylor coefficients. E.g., $y(x_0) = y_0 = F(x_0), y_1 := F'(x_0)x_1, y_2 := \frac{1}{2} F''(x_0)x_1x_1 + F'(x_0)x_2$, etc. Notice, $x_1$ and $x_2$ are input Taylor coefficients and that the $\frac{1}{2}$ comes from the fact that $\frac{d y(x_0)}{dt} = 2 y_2$. hos_reverse computes $[w^T\frac{\partial y_0}{\partial x_0}, w^T\frac{\partial y_1}{\partial x_0}, ...,w^T \frac{\partial y_d}{\partial x_0}]$

I was definitely wrong with hos_ti_reverse. Sorry for that! Maybe its called "taylor-increment", I'm not sure. However, if you set the lagrange multipliers for a degree higher than 0, it adds the results of lower degree times this lagrange multipliers to the results of a higher degree. Lets consider the following example code:

#include <adolc/adolc.h>
#include <iostream>

int main() {
  // --- 1. Create tape ---
  const short tapeId = 2;
  createNewTape(tapeId);

  trace_on(tapeId);
  {
    adouble a;
    a <<= 2.0; // independent variable

    adouble b = pow(a, 3.0); // function

    double out;
    b >>= out; // dependent variable
  }
  trace_off();

  // --- 2. Forward Taylor coefficients ---
  int dimIn = 1;
  int degre = 3;
  double x[1] = {1.0}; // base point

  auto X = myalloc2(dimIn, 3); // 1 input, 2 Taylor coefficients
  X[0][0] = 1.0;
  X[0][1] = 2.0;
  X[0][2] = 3.0;

  double y[1];
  auto Y = myalloc2(dimIn, 3); // 1 output, 2 Taylor coefficients

  hos_forward(tapeId, dimIn, dimIn, 3, 4, x, X, y, Y);
  std::cout << "1: " << Y[0][0] << std::endl;
  std::cout << "2: " << Y[0][1] << std::endl;
  std::cout << "3: " << Y[0][2] << std::endl;
  // --- 3. Prepare lagrange seeds and results ---
  int depen = 1; // 1 output
  int indep = 1; // 1 input

  auto lagrange = myalloc2(depen, degre + 1);
  for (int i = 0; i < depen; ++i) {
    lagrange[i][0] = 1.0; // 0-th order seed
    for (int j = 1; j <= degre; ++j)
      lagrange[i][j] = 0.0; // higher orders 0
  }
  lagrange[0][1] = 1.0;

  auto results = myalloc2(indep, degre + 1); // input adjoints
  for (int i = 0; i < indep; ++i)
    for (int j = 0; j <= degre; ++j)
      results[i][j] = 0.0;

  // --- 4. Reverse pass ---
  hos_ti_reverse(tapeId, depen, indep, degre, lagrange, results);

  // --- 5. Print results ---
  std::cout << "Input adjoint coefficients:" << std::endl;
  for (int k = 0; k <= degre; ++k) {
    std::cout << "results[0][" << k << "] = " << results[0][k] << std::endl;
  }

  // --- 6. Cleanup ---
  myfree2(X);
  myfree2(Y);
  myfree2(lagrange);
  myfree2(results);

  return 0;
}

If you set langrange[0][1] = 0.0 (and the others for higher degree too) you get hos_reverseas you already mentioned. If you set lagrange[0][1] = 1.0 you add to result[0][1] 1.0 result[0][0], result[0][2] is incremented by 1.0 result[0][1] and result[0][3] is incremented by 1.0 result[0][2]. If you set lagrange[0][1] = 2.5 the factor is changed accordingly. When you know set lagrange[0][1] = 0.0 and lagrange[0][2] = 1.0 you start the incrementing at result[0][2]with 1.0 result[0][0] and you also get result[0][3] incremented by 1.0 * result[0][1]. If you only set langrange[0][2] = 1.0 just the last result is incremeneted by lagrange[0][2] * result[0][0]. You can derive a precise formula for $1 \leq k \leq d$
$result_{0,k}= lagrange_{0,0}\frac{\partial y_k}{\partial x_0} + \sum_{i = 0}^{k - 1} lagrange_{0, i + 1}\frac{\partial y_{k-i-1}}{\partial x_0}$

[chrhansk]

Thanks for the explanation, that does seem to make sense. But given that this function apparently needs to be implemented to get a Hessian of a callback function it would help a lot if this was properly documented.

[TimSiebert1]

I agree. Let me see if I can add something in the next days :).