Python Environment Management

Software and Programming
1

Hello CARC folks,

I’m wondering if people have recommendations about managing python environments and packages on the cluster. I’d like to use poetry but am considering conda and virtualenv as well. My use case involves developing python software, installing and using dependencies for compute jobs, and packaging, building, and installing custom packages for use on the cluster.

In general, does anyone have advice or best practices regarding this process?

Thanks a ton,
Jon

2

It’s mostly personal preference. Any of these will work fine on the cluster. Depending on the specific Python packages you use, it may be easier to use a Conda/Mamba environment to pull in non-Python dependencies. You could also use Poetry together with Conda/Mamba.