# python – Is it possible to use pip to install a package from a private GitHub repository?

## The Question :

381 people think this question is useful

I am trying to install a Python package from a private GitHub repository. For a public repository, I can issue the following command which works fine:

pip install git+git://github.com/django/django.git



However, if I try this for a private repository:

pip install git+git://github.com/echweb/echweb-utils.git



I get the following output:

Downloading/unpacking git+git://github.com/echweb/echweb-utils.git
Cloning Git repository git://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build
Complete output from command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build:
fatal: The remote end hung up unexpectedly

Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build...

----------------------------------------
Command /usr/local/bin/git clone git://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-VRsIoo-build failed with error code 128



I guess this is because I am trying to access a private repository without providing any authentication. I therefore tried to use Git + ssh hoping that pip would use my SSH public key to authenticate:

pip install git+ssh://github.com/echweb/echweb-utils.git



This gives the following output:

Downloading/unpacking git+ssh://github.com/echweb/echweb-utils.git
Cloning Git repository ssh://github.com/echweb/echweb-utils.git to /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build
Complete output from command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build:
Cloning into /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build...

Permission denied (publickey).

fatal: The remote end hung up unexpectedly

----------------------------------------
Command /usr/local/bin/git clone ssh://github.com/echweb/echweb-utils.git /var/folders/cB/cB85g9P7HM4jcPn7nrvWRU+++TI/-Tmp-/pip-DQB8s4-build failed with error code 128



Is what I am trying to achieve even possible? If so, how can I do it?

• It certainly isn’t the correct answer, but cloning the repo manually then pointing pip to localhost instead of github will get you past this if you’re just trying to be productive.
• @nmicheals That’s what I’ve done so far, but I need to put this into requirements files for deployment across many web sites all with separate virtualenvs.
• Just to he sure: you’ve already set up ssh key support on github, right? If that’s definitely not working…. Have you tried using git+git://user:pass@github.com/…. as the URI?
• Try using eval $(ssh-agent); ssh-add ~/.ssh/github_id_rsa and then run pip installs following pip.pypa.io/en/stable/reference/pip_install/#git ## The Answer 1 412 people think this answer is useful You can use the git+ssh URI scheme, but you must set a username. Notice the git@ part in the URI: pip install git+ssh://git@github.com/echweb/echweb-utils.git  Also read about deploy keys. PS: In my installation, the “git+ssh” URI scheme works only with “editable” requirements: pip install -e URI#egg=EggName  Remember: Change the : character that git remote -v prints to a / character before using the remote’s address in the pip command: $ git remote -v
origin  git@github.com:echweb/echweb-utils.git (fetch)
#                     ^ change this to a '/' character



If you forget, you will get this error:

ssh: Could not resolve hostname github.com:echweb:
nodename nor servname provided, or not known



77 people think this answer is useful

As an additional technique, if you have the private repository cloned locally, you can do:

pip install git+file://c:/repo/directory



More modernly, you can just do this (and the -e will mean you don’t have to commit changes before they’re reflected):

pip install -e C:\repo\directory



50 people think this answer is useful

You can do it directly with the HTTPS URL like this:

pip install git+https://github.com/username/repo.git



This also works just appending that line in the requirements.txt in a Django project, for instance.

33 people think this answer is useful

It also works with Bitbucket:

pip install git+ssh://git@bitbucket.org/username/projectname.git



Pip will use your SSH keys in this case.

21 people think this answer is useful

I found it much easier to use tokens than SSH keys. I couldn’t find much good documentation on this, so I came across this solution mainly through trial and error. Further, installing from pip and setuptools have some subtle differences; but this way should work for both.

GitHub don’t (currently, as of August 2016) offer an easy way to get the zip / tarball of private repositories. So you need to point setuptools to tell setuptools that you’re pointing to a Git repository:

from setuptools import setup
import os
# Get the deploy key from https://help.github.com/articles/git-automation-with-oauth-tokens/
github_token = os.environ['GITHUB_TOKEN']

setup(
# ...
install_requires='package',
'git+https://{github_token}@github.com/user/{package}.git/@{version}#egg={package}-0'
.format(github_token=github_token, package=package, version=master)
]



A couple of notes here:

• For private repositories, you need to authenticate with GitHub; the simplest way I found is to create an OAuth token, drop that into your environment, and then include it with the URL
• You need to include some version number (here is 0) at the end of the link, even if there’s isn’t any package on PyPI. This has to be a actual number, not a word.
• You need to preface with git+ to tell setuptools it’s to clone the repository, rather than pointing at a zip / tarball
• version can be a branch, a tag, or a commit hash
• You need to supply --process-dependency-links if installing from pip

17 people think this answer is useful

I figured out a way to automagically ‘pip install’ a GitLab private repository that requires no password prompt. This approach uses GitLab “Deploy Keys” and an SSH configuration file, so you can deploy using keys other than your personal SSH keys (in my case, for use by a ‘bot). Perhaps someone kind soul can verify using GitHub.

### Create a New SSH key:

ssh-keygen -t rsa -C "GitLab_Robot_Deploy_Key"



The file should show up as ~/.ssh/GitLab_Robot_Deploy_Key and ~/.ssh/GitLab_Robot_Deploy_Key.pub.

Copy and paste the contents of the ~/.ssh/GitLab_Robot_Deploy_Key.pub file into the GitLab “Deploy Keys” dialog.

### Test the New Deploy Key

The following command tells SSH to use your new deploy key to set up the connection. On success, you should get the message: “Welcome to GitLab, UserName!”

ssh -T -i ~/.ssh/GitLab_Robot_Deploy_Key git@gitlab.mycorp.com



### Create the SSH Configuration File

Next, use an editor to create a ~/.ssh/config file. Add the following contents. The ‘Host’ value can be anything you want (just remember it, because you’ll be using it later). The HostName is the URL to your GitLab instance. The IdentifyFile is path to the SSH key file you created in the first step.

Host GitLab
HostName gitlab.mycorp.com
IdentityFile ~/.ssh/GitLab_Robot_Deploy_Key



### Point SSH to the Configuration file

oxyum gave us the recipe for using pip with SSH:

pip install git+ssh://git@gitlab.mycorp.com/my_name/my_repo.git



We just need to modify it a bit to make SSH use our new Deploy Key. We do that by pointing SSH to the Host entry in the SSH configuration file. Just replace the ‘gitlab.mycorp.com’ in the command to the host name we used in the SSH configuration file:

pip install git+ssh://git@GitLab/my_name/my_repo.git



The package should now install without any password prompt.

15 people think this answer is useful

The syntax for the requirements file is given here:

https://pip.pypa.io/en/latest/reference/pip_install.html#requirements-file-format

So for example, use:

-e git+http://github.com/rwillmer/django-behave#egg=django-behave



if you want the source to stick around after installation.

Or just

git+http://github.com/rwillmer/django-behave#egg=django-behave



if you just want it to be installed.

12 people think this answer is useful

If you want to install dependencies from a requirements file within a CI server or alike, you can do this:

git config --global credential.helper 'cache'
echo "protocol=https
host=example.com
username=${GIT_USER} password=${GIT_PASS}
" | git credential approve
pip install -r requirements.txt



pip install git+https://<username>:$PASS@github.com/echweb/echweb-utils.git  ## The Answer 12 4 people think this answer is useful If you need to do this in, say, a command line one-liner, it’s also possible. I was able to do this for deployment on Google Colab: 1. Create a Personal Access Token: https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token 2. Run: pip install git+https://<PERSONAL ACCESS TOKEN>@github.com/<USERNAME>/<REPOSITORY>.git ## The Answer 13 0 people think this answer is useful oxyum’s solution is OK for this answer. I just want to point out that you need to be careful if you are installing using sudo as the keys must be stored for root too (for example, /root/.ssh). Then you can type sudo pip install git+ssh://git@github.com/echweb/echweb-utils.git  ## The Answer 14 0 people think this answer is useful Just copy the remote from the original git clone command (or from git remote -v). You will get something like this: • Bitbucket: git+ssh://git@bitbucket.org:your_account/my_pro.git • GitHub: git+ssh://git@github.com:your_account/my_pro.git Next, you need to replace : with / next to the domain name. So install using: pip install git+ssh://git@bitbucket.org/your_account/my_pro.git  ## The Answer 15 0 people think this answer is useful Here’s a quick method that worked for me. Simply fork the repo and install it from your own GitHub account with pip install git+https://github.com/yourName/repoName  ## The Answer 16 0 people think this answer is useful My case was kind of more complicated than most of the ones described in the answers. I was the owner of two private repositories repo-A and repo-B in a Github organization and needed to pip install repo-A during the python unittests of repo-B, as a Github action. Steps I followed to solve this task: • Created a Personal Access Token for my account. As for its permissions, I only needed to keep the default ones, .i.e. repo – Full control of private repositories. • Created a repository secret under repo-B, pasted my Personal Access Token in there and named it PERSONAL_ACCESS_TOKEN. This was important because, unlike the solution proposed by Jamie, I didn’t need to explicitly expose my precious raw Personal Access Token inside the github action .yml file. • Finally, I pip installed the package from source via HTTPS as follows export PERSONAL_ACCESS_TOKEN=${{ secrets.PERSONAL_ACCESS_TOKEN }} && pip install git+https://\${PERSONAL_ACCESS_TOKEN}@github.com/MY_ORG_NAME/MY_REPO_NAME.git

-1 people think this answer is useful

If you have your own library/package on GitHub, GitLab, etc., you have to add a tag to commit with a concrete version of the library, for example, v2.0, and then you can install your package:

pip install git+ssh://link/name/repo.git@v2.0



This works for me. Other solutions haven’t worked for me.

pip install git+git@gitlab.mycorp.com/my_name/my_repo.git

without ssh:.... That works for me.