Introduce the Python naming problem and Pip’s –install-option problem.
Python package naming problem
A classic setup function:
setup(
name='myproject',
version='0.1.0',
package_dir={'':'src'},
packages=find_packages(where='src')
)
name is the project name listed in pip list. packages is the package names when using import in python code.
package_dir={'':'src'} is unprefix importing. For example:
If the project’s structure like this with the using of previous classic setup function
myproject/
│
├── src/
│ ├── common/
│ │ ├── __init__.py
│ │ └── ...
│ │
│ └── components/
│ ├── __init__.py
│ └── ...
│
└── setup.py
After installing this package, there are package common and components in Python environment. In code, they should be imported like import common and import components. However, when listing packages using pip list, it only shows myproject 0.1.0.
We can add prefix in package_dir like:
package_dir={
'myproject.common': 'src/common',
'myproject.components': 'src/components'
},
packages=find_packages(where='src')
Then, It can be imported like import myproject.common and myproject.components.
This can be consider as a naming space problem.
Pip’s –install-option problem
--install-option can pass argument to setup.py when using pip install XXX --install-option="xxx". However, it will force every dependencies package to be build from source code instead of using existing wheel files, which will lead to slow installing.
Try to avoid using it. It is simple to using environment viriables to control setup.py’s behavior. For example, controling build which Cython extension:
if os.environ.get("INSTALL_SELECTOR")==None or os.environ.get("INSTALL_SELECTOR")=="all":
compile_cpu = True
compile_gpu = True
print("Install both CPU and GPU version")
elif os.environ.get("INSTALL_SELECTOR")=="cpu":
compile_cpu = True
compile_gpu = False
print("Only install CPU version")
elif os.environ.get("INSTALL_SELECTOR")=="gpu":
compile_cpu = False
compile_gpu = True
print("Only install GPU version")
else:
raise Exception("INSTALL_SELECTOR should be in [\"all\", \"cpu\", \"gpu\"], or do not define it for \"all\". Got %s."%os.environ.get("INSTALL_SELECTOR"))
extensions = []
extensions.append(Extension(
"fastpermut.c_conf", # Name of the resulting python extension
......
))
if compile_cpu:
cpu_extension = Extension(
"fastpermut.permutation", # Name of the resulting python extension
......
)
extensions.append(cpu_extension)
if compile_gpu:
gpu_extension = Extension(
"fastpermut.permutation_cuda", # Name of the resulting python extension
......
)
extensions.append(gpu_extension)
setup(
......
ext_modules=cythonize(extensions, language_level=3),
)
