Picoquic builds failing because of module dependency

[huitema]

The picoquic builds on travis align on specific commits of picotls, using command:

# Build at a known-good commit
COMMIT_ID=4e6080b6a1ede0d3b23c72a8be73b46ecaf1a084

cd ..
git clone --branch master --single-branch --shallow-submodules --recurse-submodules --no-tags https://github.com/h2o/picotls

The specific commit ID was copied from the git log:

Here is the report from the picoquic travis build:

Cloning into 'picotls'...
remote: Enumerating objects: 101, done.
remote: Counting objects: 100% (101/101), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 6489 (delta 58), reused 82 (delta 48), pack-reused 6388
Receiving objects: 100% (6489/6489), 4.45 MiB | 16.52 MiB/s, done.
Resolving deltas: 100% (3747/3747), done.
Submodule 'deps/picotest' (https://github.com/h2o/picotest.git) registered for path 'deps/picotest'
Cloning into '/home/travis/build/private-octopus/picotls/deps/picotest'...
remote: Enumerating objects: 4, done.        
remote: Counting objects: 100% (4/4), done.        
remote: Compressing objects: 100% (4/4), done.        
remote: Total 4 (delta 0), reused 2 (delta 0), pack-reused 0        
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
error: Server does not allow request for unadvertised object f390562fd4d6919807441721ec05b08f6d8c8d9c
Fetched in submodule path 'deps/picotest', but it did not contain f390562fd4d6919807441721ec05b08f6d8c8d9c. Direct fetching of that commit failed.
M	deps/picotest
Note: checking out '4e6080b6a1ede0d3b23c72a8be73b46ecaf1a084'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

Any idea how to solve that?

[kazuho]

Looks like a temporary? glitch in GitHub.

The error is error: Server does not allow request for unadvertised object f390562fd4d6919807441721ec05b08f6d8c8d9c, even though the commit exists here: h2o/picotest@f390562.

[huitema]

Github is protesting that the commit exists for picotls, but does not exist for deps/picotest. Do you have a commit ID that is verified to exist for picotest?

[huitema]

OK, I did verify that the commit mentioned in github is visible in git log for picotest. This is really weird.

[kazuho]

Does rerunning the test solve the issue? Hopefully it's a temporary error in the HTTP cache.

[huitema]

I tried re-running several times. Same error.

[huitema]

It is frustrating because I don't actually need the test code. But there is no cmake option to not produce the test code.

[kazuho]

Can you update picotls? Up-to-date master branch depends on a different commit of picotest :-)

[huitema]

This is done in a script, starting from a clean state. I don't understand what "update picotls" means here. Do you mean before the git clone ... command?

[kazuho]

@huitema My bad. I thought I changed the commit of picotest that picotls refers to. However, that was the case for quicly, not picotls.

I've updated the ref of deps/picotest in commit 7898a8d. If you update your reference of picotls to up-to-date master, then you'd no longer be loading the problematic commit of picotls (i.e.
h2o/picotest@f390562).

[huitema]

Synching on the latest commit in picotls did fix the issue. Thanks.

[huitema]

Still failing in more recent builds:

make --version
cmake version 3.12.4
CMake suite maintained and supported by Kitware (kitware.com/cmake).
before_script.1
6.81s$ ./ci/build_picotls.sh
Cloning into 'picotls'...
remote: Enumerating objects: 101, done.
remote: Counting objects: 100% (101/101), done.
remote: Compressing objects: 100% (71/71), done.
remote: Total 6631 (delta 59), reused 60 (delta 30), pack-reused 6530
Receiving objects: 100% (6631/6631), 4.56 MiB | 16.81 MiB/s, done.
Resolving deltas: 100% (3840/3840), done.
Submodule 'deps/picotest' (https://github.com/h2o/picotest.git) registered for path 'deps/picotest'
Cloning into '/home/travis/build/private-octopus/picotls/deps/picotest'...
remote: Enumerating objects: 4, done.        
remote: Counting objects: 100% (4/4), done.        
remote: Compressing objects: 100% (4/4), done.        
remote: Total 4 (delta 0), reused 2 (delta 0), pack-reused 0        
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
error: Server does not allow request for unadvertised object f390562fd4d6919807441721ec05b08f6d8c8d9c
Fetched in submodule path 'deps/picotest', but it did not contain f390562fd4d6919807441721ec05b08f6d8c8d9c. Direct fetching of that commit failed.
error: Server does not allow request for unadvertised object f390562fd4d6919807441721ec05b08f6d8c8d9c
Fetched in submodule path 'deps/picotest', but it did not contain f390562fd4d6919807441721ec05b08f6d8c8d9c. Direct fetching of that commit failed.

@kazuho Our scripts load picotls using:

COMMIT_ID=d18549f4586a43ec5fadbbdd1d0aa640a2bd75fe
git clone --branch master --single-branch --shallow-submodules --recurse-submodules --no-tags https://github.com/h2o/picotls
cd picotls
git checkout -q "$COMMIT_ID"

It obviously does not work. Are we doing something wrong?