rethink nydusd configuration file

[imeoer]

1. Nydusd configuration file includes registry auth information, it shouldn't be saved to disk (how to handle live upgrade and failover?).
2. When the nydusd configuration file changes, the new nydusd should use the updated.
3. Once the auth subscribed from k8s secret/docker config changes, nydusd also needs to update, in case of using an expired auth.

[changweige]

When the nydusd configuration file changes, the new nydusd should use the updated.

nydus-snapshotter is providing the new nydusd config file to nydusd now

[DarkMountain-wyz]

1. Nydusd configuration file includes registry auth information, it shouldn't be saved to disk (how to handle live upgrade and failover?).
2. When the nydusd configuration file changes, the new nydusd should use the updated.
3. Once the auth subscribed from k8s secret/docker config changes, nydusd also needs to update, in case of using an expired auth.

I am Guijie Wang, and I will complete this part of the work.

[changweige]

When the nydusd configuration file changes, the new nydusd should use the updated.

The already running nydusd should not change its configuration I suppose. The newly created nydusd can use the new nydusd configuration. Otherwise, it's not easy to handle the consistency between nydusd config file and nydusd DB records

Can we only add an interface or credential service to nydus-snapshotter? The credential service can provide auth to nydusd when it requests

[imeoer]

The already running nydusd should not change its configuration I suppose. The newly created nydusd can use the new nydusd configuration. Otherwise, it's not easy to handle the consistency between nydusd config file and nydusd DB records

Yes, we said "the new nydusd should use the updated.".

Can we only add an interface or credential service to nydus-snapshotter? The credential service can provide auth to nydusd when it requests

The registry credential is the one of wanting to be updated for nydusd, the new nydusd maybe also need the new prefetch configuration, etc. I think we'd better not let nydusd become aware of the credential, it should be handled by the containerd credential plugin with snapshotter in the future, and then nydusd only get the full configuration from snapshotter.

[changweige]

The already running nydusd should not change its configuration I suppose. The newly created nydusd can use the new nydusd configuration. Otherwise, it's not easy to handle the consistency between nydusd config file and nydusd DB records

Yes, we said "the new nydusd should use the updated.".

Can we only add an interface or credential service to nydus-snapshotter? The credential service can provide auth to nydusd when it requests

The registry credential is the one of wanting to be updated for nydusd, the new nydusd maybe also need the new prefetch configuration, etc. I think we'd better not let nydusd become aware of the credential, it should be handled by the containerd credential plugin with snapshotter in the future, and then nydusd only get the full configuration from snapshotter.

The registry auth ever passed to nydusd might be expired due to the registry configuration. So I suppose we need a way to refresh the auth nydusd is using.

[changweige]

The already running nydusd should not change its configuration I suppose. The newly created nydusd can use the new nydusd configuration. Otherwise, it's not easy to handle the consistency between nydusd config file and nydusd DB records

Yes, we said "the new nydusd should use the updated.".

Can we only add an interface or credential service to nydus-snapshotter? The credential service can provide auth to nydusd when it requests

The registry credential is the one of wanting to be updated for nydusd, the new nydusd maybe also need the new prefetch configuration, etc. I think we'd better not let nydusd become aware of the credential, it should be handled by the containerd credential plugin with snapshotter in the future, and then nydusd only get the full configuration from snapshotter.

Moreover, I don't think the prefetch configurations have to update for an ever-started nydusd which had finished its prefetch jobs.

[bergwolf]

3. Once the auth subscribed from k8s secret/docker config changes, nydusd also needs to update, in case of using an expired auth.

Can we make nydus-snapshotter to send the updates to nydusd via a (new) nydusd API? Then there is no need for nydusd to connect to snapshotter UDS.