Cargo Auth for Private Git Repos

If you use Cargo with private git repositories, you've probably run into issues caused by the fact that Cargo doesn't shell out to git to fetch repositories, and instead uses the git2 crate, which depends on C deps libgit2 and libssh2. git2 and libssh2 do not read https and ssh authentication exactly the same way that git and OpenSSH do, which can lead to things not working out of the box. See rust-lang/cargo#1851 for more information.

Workaround

The easy workaround is to set an environment variable to tell Cargo to shell out to git instead of using git2, by setting export CARGO_NET_GIT_FETCH_WITH_CLI=true. You can also add this to your config with:

# Add to $CARGO_HOME/config , which is usually ~/.cargo/config
[net]
git-fetch-with-cli = true

If you do this, then anything that you can fetch with git should work directly with Cargo. However there are many cases where you don't want to depend on having Git in the $PATH, and would rather just have Cargo work by itself.

Fix

The below fixes are fairly easy, but they assume you already have git clone working for the URL you are trying to add to Cargo, so check that is working first.

HTTPS

For HTTPS URLs, Cargo uses Git credentials to authenticate.

You can check if your credentials are set up correctly by running:

# Change to your base URL if different, e.g. github.mycompany.com
GIT_HOST=github.com
GIT_PATH=org/repo

echo "protocol=https
host=${GIT_HOST?}
path=${GIT_PATH?}" | git credential fill

This should print out your OAuth token for that host. If it doesn't work you'll need to add it.

macOS

On macOS the setup is fairly straightforward, as you can store your credentials for each host in the System Keychain, and then have Cargo read it from there.

To tell Git to store credentials in the system keychain run:

git config --global credential.helper osxkeychain

Then you need to add your User and Password to the keychain. See the GitHub OAuth Token docs for information on generating a token for GitHub, other providers will support the same functionality.

GIT_USER=<your username>
GIT_TOKEN=<your oauth token>
# Change to your base URL if different, e.g. github.mycompany.com
GIT_HOST=github.com

echo "protocol=https
host=${GIT_HOST?}
username=${GIT_USER?}
password=${GIT_TOKEN?}" | git credential-osxkeychain store

Linux

For Linux the situation is a bit more varied, as there is no guarantee you have a built-in system keychain. Using the in-memory credential cache built into Git should work everywhere though:

# Tell Git to use the cache credential helper, and cache for 3600 seconds (60 minutes).
# Default is 900 seconds (15 minutes).
git config --global credential.helper 'cache --timeout=3600'

See git-credential-cache for more information, and see git-credential-store to store passwords in a file on-disk (more risky but more convenient).

You may also be able to find a Credential Helper for your platform, see StackOverflow cache GitHub Credentials for more information.

Without Git

This is a bit harder, but luckily cargo will parse a Git Config file even without git installed. You simply need to add a valid config to one of the Git Config locations that tells cargo how to get your username and password.

For example, if you have your username and password in the following file at say ~/.secret:

username=<user>
password=<password>

you could run this command to set up your config correctly:

echo -e '[credential]\nhelper = "!cat ~/.secret; : "' > ~/.gitconfig

Cargo will call your helper command like so to generate the credentials:

# user line is optional, probably not included.
echo "protocol=https
host=github.com
user=myusername
" | /bin/sh -c '<command> get'

so you need to ensure you ignore or handle the get argument. For more complicated use-cases you probably want to shell out to an executable (a script or binary that will return the required credentials). For example you may need to return different credentials depending on the host you receive on the stdin.

You can test that your command works by enabling debug logs (which will show you the execution of the credential helper script).

CARGO_LOG=git2=debug cargo build

SSH

For SSH if you can git clone your repository then Cargo should be able to fetch it too, there are two common problems.

Adding SSH Keys to Agent

If you have told Git/OpenSSH where to find your SSH Keys for the github URL by adding to your SSH Config, you may need to load them manually. You can confirm this by running:

grep -i IdentityFile ~/.ssh/config | sort -u

If this prints IdentityFiles you will need to make sure those are manually added to your ssh-agent, as libssh2 does not parse your ~/.ssh/config.

If you use the default SSH Key paths, you can simply run this to add them to the ssh-agent:

# On a Mac:
ssh-add --apple-use-keychain --apple-load-keychain

# Not on a Mac:
ssh-add

To automatically add them you can follow this use macOS keychain with SSH Keys post and prepend this to your ~/.ssh/config file:

Host *
    UseKeychain yes
    AddKeysToAgent yes

Custom Key Paths

If you use non-default paths, provide the paths as arguments to ssh-add, using the paths from the 'grep` command above, e.g.

# On a Mac:
ssh-add -AK ~/.ssh/gibfahn_id_ed25519 ~/.ssh/gibfahn_id_rsa

# Not on a Mac:
ssh-add ~/.ssh/gibfahn_id_ed25519 ~/.ssh/gibfahn_id_rsa

GitHub SSH URL Syntax

Cargo (via libssh2) only supports "explicit" ssh syntax, which means that ssh URLs you get from GitHub or GitHub Enterprise might not work out of the box. If so you can modify the URL by prepending ssh:// and changing the : to a /.

# Private GitHub Repo:
-git@github.com:org/repo.git
+ssh://git@github.com/org/repo.git

# GitHub Enterprise Repo:
-git@github.mycompany.com:org/repo.git
+ssh://git@github.mycompany.com/org/repo.git

Git InsteadOf

If you use Golang with private repos, you probably have an insteadOf configured in your git config. If you need to make the above change to that config, you can run git config --edit --global to open the config, and then make the following changes:

# Private GitHub Repo:
-[url "git@github.com:"]
+[url "ssh://git@github.com/"]
  insteadOf = https://github.com/

# GitHub Enterprise Repo:
-[url "git@github.mycompany.com:"]
+[url "ssh://git@github.mycompany.com/"]
  insteadOf = https://github.mycompany.com/