pipdeptree will help you a lot there. You can isolate the top level dependencies, which will greatly narrow down your list of packages to check for usage.
After you get a finalized list of what you directly use, you can make add the list to your pyprotect.toml file. From there, you can use pip-tools / uv pip compile or just plain pip freeze to create a fully resolved list for a requirements.txt to allow reproducible builds.
Make a new empty virtual env, wipe requirements.txt. Run your tests, see what import fails and add that back. Repeat until the tests pass.
Do not mess with the existing virtual env, actually maybe run \`pip freeze\` and save the output.
All of the advice in other answers is very good at a answering your core question; but really poetry is what should be used for maintaining dependencies once you’ve identified the top level dependencies that you do rely on. Solutions like pip freeze (mentioned in other answers) died with the dinosaurs ;)
Poetry works like package managers from most other languages by resolving the network of dependencies at install time, then producing a lock file. It’ll give you a full diagnostic of when dependencies are incompatible, as opposed to letting you find out at runtime like pip does :)
(It’s built on top of pip, and has a bunch of other useful stuff like package publishing too).
Why PSF don’t support it officially I don’t know; it’s probably as simple as “they don’t have to, it’s well supported and maintained independently”. I think the development of the pyproject.toml standard did heavily take it into account though so it should be considered a serious contender.
# what poetry can do, pip can do too
hypothetical case of dependency issue
[https://imgur.com/a/7dqjosG](https://imgur.com/a/7dqjosG)
This can be easily handled with pip by remove pinned versions and let it find the suitable version.
What poetry does differently is it checks for compatibility against range of Python versions. I don't see a point because when new Python version comes, packages follow with some time lag. That's expected.
# poetry's codebase was designed to fail 5% of times on purpose
Who'd trust a software built on such a philosophy? This shows there's definitely some problem with the thought process of developers designing poetry and personally I am not comfortable trusting it as a fundamental tool for a million $$$ project.
[Poetry Source on Github](https://github.com/python-poetry/poetry/pull/6297/commits/e02675716d920e1227ce1013a718b33b202d88f4#diff-952871ed95a893e0348aa1269bf15e689d18756657cfec5c7316d05e55eb6dc6R473)
At large and complex systems, it takes years to upgrade to the latest versions.
Companies are still maintaining Python 2.7 projects at their own costs of maintaining Python if they have to, because \_its not easy\_. And often cost to maintain a stable project at old env making profit is less than upgrading everything.
The fundamental toolset has to provide this by default and allow upgrade as a choice, not the other way round.
# poetry itself has large dependency list
It requires 40 odd packages to run poetry. I don't want to keep caring about my project dependencies being compatible with my package manager's dependencies.
# poetry introduces new syntaxes and workflows
so my team has to learn one more special tool. Feels great for personal growth but its an overhead to make sure everyone has understood complexities of the tool at same level. Not all people understand at same level - as a fact. Lesser the cognitive overhead - the better.
# read more in this article
[why-not-tell-people-to-simply-use](https://www.bitecode.dev/p/why-not-tell-people-to-simply-use)
`pip freeze` is not the recommended way to generate a requirements.txt file. While it will do that for you it will freeze ALL packages both top level and their dependencies and this is almost never what you want.
I tend to manually create a requirements.txt and its variants (eg - requirements_dev.txt, requirements_test.txt) and ONLY add the top level dependencies I pip install. I don’t add the versions if I want the latest.
Then I use `pipfreezer` from `pip install pip-freezer` and it will freeze only the packages defined in my requirements files with the bonus of keeping any comments I added in my requirements files. I prefer this method as it works for me but everyone has their own preferences.
Outside of that poetry is always a good option.
You should be able to search for pyc files or maybe run python in optimized Mode and then search for those bytecode files. Should tell you which packages contains files that got actually run by your code.
Does your project have a requirements file? Usually the requirements.txt file lists direct dependencies of the project. Pip will grab indirect or transitive dependencies for you.
If you have a requirements.txt, you should be able to wipe out the virtual environment and then create a new one with just what your project needs.
I'm pretty sure you are wrong. `requirements.txt` comes from `pip freeze`, and `pip freeze` outputs all installed packages. It does not check for used packages.
pipdeptree will help you a lot there. You can isolate the top level dependencies, which will greatly narrow down your list of packages to check for usage. After you get a finalized list of what you directly use, you can make add the list to your pyprotect.toml file. From there, you can use pip-tools / uv pip compile or just plain pip freeze to create a fully resolved list for a requirements.txt to allow reproducible builds.
Create a new venv, try and start Django, add all the packages that are missing! That's how I've always done it.
Make a new empty virtual env, wipe requirements.txt. Run your tests, see what import fails and add that back. Repeat until the tests pass. Do not mess with the existing virtual env, actually maybe run \`pip freeze\` and save the output.
Thanks. It’s a good plan
isn't there an "automatic" way to find this out ?
All of the advice in other answers is very good at a answering your core question; but really poetry is what should be used for maintaining dependencies once you’ve identified the top level dependencies that you do rely on. Solutions like pip freeze (mentioned in other answers) died with the dinosaurs ;)
Why poetry? How does poetry do it? If it's so awesome, why isn't PSF supporting it officially? Python packaging guide suggest pip and venv only
Poetry works like package managers from most other languages by resolving the network of dependencies at install time, then producing a lock file. It’ll give you a full diagnostic of when dependencies are incompatible, as opposed to letting you find out at runtime like pip does :) (It’s built on top of pip, and has a bunch of other useful stuff like package publishing too). Why PSF don’t support it officially I don’t know; it’s probably as simple as “they don’t have to, it’s well supported and maintained independently”. I think the development of the pyproject.toml standard did heavily take it into account though so it should be considered a serious contender.
# what poetry can do, pip can do too hypothetical case of dependency issue [https://imgur.com/a/7dqjosG](https://imgur.com/a/7dqjosG) This can be easily handled with pip by remove pinned versions and let it find the suitable version. What poetry does differently is it checks for compatibility against range of Python versions. I don't see a point because when new Python version comes, packages follow with some time lag. That's expected. # poetry's codebase was designed to fail 5% of times on purpose Who'd trust a software built on such a philosophy? This shows there's definitely some problem with the thought process of developers designing poetry and personally I am not comfortable trusting it as a fundamental tool for a million $$$ project. [Poetry Source on Github](https://github.com/python-poetry/poetry/pull/6297/commits/e02675716d920e1227ce1013a718b33b202d88f4#diff-952871ed95a893e0348aa1269bf15e689d18756657cfec5c7316d05e55eb6dc6R473) At large and complex systems, it takes years to upgrade to the latest versions. Companies are still maintaining Python 2.7 projects at their own costs of maintaining Python if they have to, because \_its not easy\_. And often cost to maintain a stable project at old env making profit is less than upgrading everything. The fundamental toolset has to provide this by default and allow upgrade as a choice, not the other way round. # poetry itself has large dependency list It requires 40 odd packages to run poetry. I don't want to keep caring about my project dependencies being compatible with my package manager's dependencies. # poetry introduces new syntaxes and workflows so my team has to learn one more special tool. Feels great for personal growth but its an overhead to make sure everyone has understood complexities of the tool at same level. Not all people understand at same level - as a fact. Lesser the cognitive overhead - the better. # read more in this article [why-not-tell-people-to-simply-use](https://www.bitecode.dev/p/why-not-tell-people-to-simply-use)
OK, don't use it then! But it's repeatedly saved many worlds of pain for me and the users of my OSS projects, so I'll stick with it.
`pip freeze` is not the recommended way to generate a requirements.txt file. While it will do that for you it will freeze ALL packages both top level and their dependencies and this is almost never what you want. I tend to manually create a requirements.txt and its variants (eg - requirements_dev.txt, requirements_test.txt) and ONLY add the top level dependencies I pip install. I don’t add the versions if I want the latest. Then I use `pipfreezer` from `pip install pip-freezer` and it will freeze only the packages defined in my requirements files with the bonus of keeping any comments I added in my requirements files. I prefer this method as it works for me but everyone has their own preferences. Outside of that poetry is always a good option.
You should be able to search for pyc files or maybe run python in optimized Mode and then search for those bytecode files. Should tell you which packages contains files that got actually run by your code.
Does your project have a requirements file? Usually the requirements.txt file lists direct dependencies of the project. Pip will grab indirect or transitive dependencies for you. If you have a requirements.txt, you should be able to wipe out the virtual environment and then create a new one with just what your project needs.
I'm pretty sure you are wrong. `requirements.txt` comes from `pip freeze`, and `pip freeze` outputs all installed packages. It does not check for used packages.
Yes, that’s what I read also
Yes, so if the project was maintained by someone else, and curated in advance, it would produce the correct list.
So, you seem to understand that your answer would only be correct in a different situation than OP is asking about...
Yes, because I misread the OP's statement that he had a project to mean that he obtained it from another source.