python – Automatically create requirements.txt

The Question :

376 people think this question is useful

Sometimes I download the python source code from github and don’t know how to install all the dependencies. If there is no requirements.txt file I have to create it by hands. The question is: Given the python source code directory is it possible to create requirements.txt automatically from the import section?

The Question Comments :
  • You can do it by running pip freeze > requirements.txt will save all your python libraries with current version into requirements.txt file
  • @Shaikhul but that doesn’t help in the case where you don’t have the dependencies installed because you’ve only just downloaded the package from GitHub…
  • Please note: a) pip freeze will dump current versions of all the installed modules on that system irrespective of there usage in the current project. b) pip will only list modules that have been installed via pip
  • Possible duplicate of How to create a requirements.txt?

The Answer 1

409 people think this answer is useful

If you use virtual environment, pip freeze > requirements.txt just fine. IF NOT, pigar will be a good choice for you.

By the way, I do not ensure it will work with 2.6.

UPDATE:

Pipenv or other tools is recommended for improving your development flow.

For Python 3 use below

pip3 freeze > requirements.txt

The Answer 2

485 people think this answer is useful

You can use the following code to generate a requirements.txt file:

pip install pipreqs

pipreqs /path/to/project

more info related to pipreqs can be found here.

Sometimes you come across pip freeze, but this saves all packages in the environment including those that you don’t use in your current project.

The Answer 3

17 people think this answer is useful

In my case, I use Anaconda, so running the following command from conda terminal inside my environment solved it, and created this requirements txt file for me automatically:

conda list -e > requirements.txt

This was taken from this Github link pratos/condaenv.txt

If an error been seen, and you are using anaconda, try to use the .yml option:

conda env export > <environment-name>.yml

For other person to use the environment…Or if you are creating a new enviroment on other machine: conda env create -f .yml

.yml option been found here

The Answer 4

10 people think this answer is useful

Make sure to run pip3 for python3.7.

pip3 freeze >> yourfile.txt

Before executing the above command make sure you have created a virtual environment.

python3:

pip3 install virtualenv
python3 -m venv <myenvname> 

python2:

pip install virtualenv
virtualenv <myenvname>

After that put your source code in the directory. If you run the python file now, probably It won’t launch If you are using non-native modules. You can install those modules runing

pip3 install <module> or pip install <module> 

This will not affect you entire module list except the environment you are In.

Now you can execute the command at the top and now you have a requirements file which contains only the modules you installed in the virtual environment. Now you can run the command at the top.

I advise everyone to use environments as It makes things easier when It comes to stuff like this.

Hope this helped.

The Answer 5

6 people think this answer is useful

best way for Python 3 is:

pip3 freeze > requirements.txt

it worked for me…

The Answer 6

5 people think this answer is useful

If Facing the same issue as mine i.e. not on the virtual environment and wants requirements.txt for a specific project or from the selected folder(includes children) and pipreqs is not supporting.

You can use :

import os
import sys
from fuzzywuzzy import fuzz
import subprocess

path = "C:/Users/Username/Desktop/DjangoProjects/restAPItest"


files = os.listdir(path)
pyfiles = []
for root, dirs, files in os.walk(path):
      for file in files:
        if file.endswith('.py'):
              pyfiles.append(os.path.join(root, file))

stopWords = ['from', 'import',',','.']

importables = []

for file in pyfiles:
    with open(file) as f:
        content = f.readlines()

        for line in content:
            if "import" in line:
                for sw in stopWords:
                    line = ' '.join(line.split(sw))

                importables.append(line.strip().split(' ')[0])

importables = set(importables)

subprocess.call(f"pip freeze > {path}/requirements.txt", shell=True)

with open(path+'/requirements.txt') as req:
    modules = req.readlines()
    modules = {m.split('=')[0].lower() : m for m in modules}


notList = [''.join(i.split('_')) for i in sys.builtin_module_names]+['os']

new_requirements = []
for req_module in importables:
    try :
        new_requirements.append(modules[req_module])

    except KeyError:
        for k,v in modules.items():
            if len(req_module)>1 and req_module not in notList:
                if fuzz.partial_ratio(req_module,k) > 90:
                    new_requirements.append(modules[k])

new_requirements = [i for i in set(new_requirements)]

new_requirements

with open(path+'/requirements.txt','w') as req:
    req.write(''.join(new_requirements))

P.S: It may have a few additional libraries as it checks on fuzzylogic.

The Answer 7

1 people think this answer is useful

Not a complete solution, but may help to compile a shortlist on Linux.

grep --include='*.py' -rhPo '^\s*(from|import)\s+\w+' . | sed -r 's/\s*(import|from)\s+//' | sort -u > requirements.txt

The Answer 8

0 people think this answer is useful

Firstly, your project file must be a py file which is direct python file. If your file is in ipynb format, you can convert it to py type by using the line of code below:

jupyter nbconvert --to=python

Then, you need to install pipreqs library from cmd (terminal for mac).

pip install pipreqs

Now we can create txt file by using the code below. If you are in the same path with your file, you can just write ./ . Otherwise you need to give path of your file.

pipreqs ./

or

pipreqs /home/project/location

That will create a requirements.txt file for your project.

The Answer 9

0 people think this answer is useful

I blindly followed the accepted answer of using pip3 freeze > requirements.txt

It generated a huge file that listed all the dependencies of the entire solution, which is not what I wanted.

So you need to figure out what sort of requirements.txt you are trying to generate.

If you need a requirements.txt file that has ALL the dependencies, then use the pip3

pip3 freeze > requirements.txt

However, if you want to generate a minimal requirements.txt that only lists the dependencies you need, then use the pipreqs package. Especially helpful if you have numerous requirements.txt files in per component level in the project and not a single file on the solution wide level.

pip install pipreqs
pipreqs [path to folder]
e.g. pipreqs .

The Answer 10

-2 people think this answer is useful

if you are using PyCharm, when you open or clone the project into the PyCharm it shows an alert and ask you for installing all necessary packages.

Add a Comment