External with authentication

[inquisitev]

Is your feature request related to a problem? Please describe.

I cannot download the archive of a github branch when it requires authentication.

Describe the solution you'd like

I would like a way to use an authenticated way to pull the archive for a github repo.
even if we were given a custom retrieval command, that would work , in that case i would probably do a gh api query where authentication is already taken care of

Describe alternatives you've considered

I have tried this

which i though worked once, but on another computer it does not work( i don't think it actually ever worked now, but that i did something on the last pc ). I must use SSH because my organization does not allow http :(

I see that the docs only refer to http forms. so im guessing this is ignored because the url doesnt match. I think it should allow ssh forms though, consididering i can apply based on the ssh form on initial setup.

When i run apply, it seems to completely skip this file. some feedback there would be nice. even verbose makes me think the file is ignored completely

[inquisitev]

now after writing this issue it suddenly installs the folder, this is with the SSH form that i had before experimenting this morning, but only once?

[KapJI]

Putting password directly to externals file doesn't seem to be a good idea. It's much better to use ssh-agent for this: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#adding-your-ssh-key-to-the-ssh-agent

Or you can just add entry in ~/.ssh/config with path to your private key.

[inquisitev]

Putting password directly to externals file doesn't seem to be a good idea. It's much better to use ssh-agent for this: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#adding-your-ssh-key-to-the-ssh-agent

Or you can just add entry in ~/.ssh/config with path to your private key.

Nobody is suggesting that we put a password in the externals file? Obviously, that is a bad idea.

From what i can tell, there is no way to download an archive type with authentication. if i had a custom cmd arg that i could do some gh api xxxx command that would download the tar.gz. (no password required because its setup before and will use the ssh key by default if setup with ssh correctly)

maybe the redactions were confusing, the only stuff that was redacted was the enterprise URL and my account ID. no passwords... :)

[KapJI]

if i had a custom cmd arg that i could do some gh api xxxx command

I think you can put access token to one of these env vars:

"CHEZMOI_GITHUB_ACCESS_TOKEN",
"CHEZMOI_GITHUB_TOKEN",
"GITHUB_ACCESS_TOKEN",
"GITHUB_TOKEN",

Also will #3748 help for your use case?

[inquisitev]

interesting, With the access key i can atleast get wget to download the zip. that MIGHT work, but then i would have to find a way to inject that safely.

Im not sure what the scope of #3748 is, i would not expect that this would be captured from the comments i read

[twpayne]

I see that the docs only refer to http forms. so im guessing this is ignored because the url doesnt match. I think it should allow ssh forms though, consididering i can apply based on the ssh form on initial setup.

The url in the external's configuration is passed unmodified to git clone, so you can use SSH repo URL, or any other URL type that git clone accepts.

What are you providing for authentication? If it's a SSH private key, then you can install that manually (or have chezmoi retrieve it from your password manager or encrypted file). If you are providing an access token in one of the environment variables listed in #4120 (comment).

[inquisitev]

Im fine with using the git clone style, but it seems to just not do anything most of the time for me :(

Right here, i remove the private dotfiles folder, chezmoi apply (after the refresh period is up, same result with the --refresh-externals). If the repo is already there, it will take extra time as if it is doing a git pull. If its not already clone, it does NOTHING. -v adds no information.

authentication in this case is already setup with git as i hope the example shows. otherwise i was doing https::@github.xxx/xxx/xxx.com as a test, no dice ...

[twpayne]

Please run the command with --debug. This will show you exactly what chezmoi is doing.

[inquisitev]

Please run the command with --debug. This will show you exactly what chezmoi is doing.

here we go. Seems like it fails to lstat the folder

[twpayne]

Please include the full output of the command with --debug.

[inquisitev]

Here is the full log, what is the right way to capture this to a text file?

xxx --debug > out.txt did not capture the logging.

[twpayne]

Here is the full log, what is the right way to capture this to a text file?

xxx --debug > out.txt did not capture the logging.

Redirect stderr to the file with:

xxx --debug 2> out.txt

The error in your screenshot looks like a file not found error. Note that chezmoi apply applies files in alphabetical order (see application order for more details), which means that chezmoi will populate ~/.bashrc before ~/.local/share/private-dotfiles.

[inquisitev]

so .chezmoiexternals will not be evaluated on until after .bashrc is evaluated?

That seems very weird to me. I assumed that it would be used as a setup, so it should run before the others. If this is the case, can you recommend a work around? am i to add a run_once script that clones the repo?

[twpayne]

so .chezmoiexternals will not be evaluated on until after .bashrc is evaluated?

No. See the application order. .chezmoiexternals is read as part of the source state, but ~/.local/share/private-dotfiles will only be created after ~/.bashrc, and it looks like your ~/.bashrc template needs ~/.local/share/private-dotfiles to be created before.

[inquisitev]

ok sure, how do i make that happen? because the folder private-dotfiles is managed by chezmoi with the chezmoi externals. do i need to rename the file to change the order? looking for some ideas at this point.

I can source the private stuff in my bash rc if it exists instead of doing the chezmoi template injecting but that seems like a limited work around.

[twpayne]

ok sure, how do i make that happen? because the folder private-dotfiles is managed by chezmoi with the chezmoi externals. do i need to rename the file to change the order? looking for some ideas at this point.

tl;dr you're encountering problems because you're not using chezmoi the way it's designed to be used.

chezmoi is designed to generate your dotfiles for the current machine from a single source of truth, i.e. a single branch from a single git repo. You're trying to generate dotfiles from multiple repos.

To keep the private parts of your dotfiles secret, use a password manager or encryption. Don't use a separate repo.