How to properly package set of callable python scripts or modules

user_2857338 0
Viewed 963

I've been searching the net for quite some time now but I can't seem to wrap my head around on how can I distribute my python scripts for my end user.

I've been using my scripts on my command line using this command python samplemodule.py "args1"

And this is also the way I want my user to also use it on their end with their command line. But my worry is that this certain modules have dependencies on other library or modules.

My scripts are working when they are all in the Project's root directory, but everything crumbles when I try to package them and put them in sub directories.

An example of this is I can't now run my scripts since its having an error when I'm importing a module from the data subdirectory.

This is my project structure.

MyProject
    \formatter
      __init__.py
      __main__.py
      formatter.py
      addfilename.py
      addscrapertype.py
     ...\data
         __init__.py
         helper.py
     csv_formatter.py
     setup.py

The csv_formatter.py file is just a wrapper to call the formatter.main.

Update: I was now able to generate a tar.gz package but the package wasn't callable when installed on my machine.

This is the setup.py:

import setuptools

with open("README.md", "r") as fh:
    long_description = fh.read()

setuptools.setup(
    name="formatter",
    version="1.0.1",
    author="My Name",
    author_email="sample@email.com",
    description="A package for cleaning and reformatting csv data",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://github.com/RhaEL012/Python-Scripts",
    packages=["formatter"],
    include_package_data=True,
    package_data={
    # If any package contains *.txt or *.rst files, include them:
        "": ["*.csv", "*.rst", "*.txt"],
    },
    entry_points={
         "console_scripts": [
            "formatter=formatter.formatter:main"
        ]
    },
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires='>=3.6',
    install_requires=[
        "pandas"
    ]
)

Now, after installing the package on the machine I wasn't able to call the module and it results in an error:

Z:\>addfilename "C:\Users\Username\Desktop\Python Scripts\"

Error Message

Update: I try to install the setup.py in a virtual environment just to see where the error is coming from.

I install it then I get the following error: FileNotFoundError: [Errno 2] no such file or directory: 'README.md'

I try to include the README.md in the MANIFEST.in but still no luck. So I try to make it a string just to see if the install will proceed.

The install proceed but then again, I encounter an error that says that the package directory 'formatter' does not exist

pyinstaller python-3.x package python
3 Answers

As I am not able to look into your specific files I will just explain how I usually tackle this issue.

This is the manner how I usually setup the command line interface (cli) tools. The project folder looks like: 

Projectname
├── modulename
│   ├── __init__.py # this one is empty in this example
│   ├── cli
│   │   ├── __init__.py # this is the __init__.py that I refer to hereafter
│   ├── other_subfolder_with_scripts
├── setup.py

Where all functionality is within the modulename folder and subfolders. In my __init__.py I have:

def main():
    # perform the things that need to be done 
    # also all imports are within the function call
    print('doing the stuff I should be doing')

but I think you can also import what you want into the __init__.py and still reference to it in the manner I do in setup.py. In setup.py we have:

import setuptools

setuptools.setup(
    name='modulename',
    version='0.0.0',
    author='author_name',
    packages=setuptools.find_packages(),
    entry_points={
        'console_scripts': ['do_main_thing=modulename.cli:main'] # so this directly refers to a function available in __init__.py
        },
)

Now install the package with pip install "path to where setup.py is" Then if it is installed you can call: 

do_main_thing
>>> doing the stuff I should be doing

For the documentation I use: https://setuptools.readthedocs.io/en/latest/.

My recommendation is to start with this and slowly add the functionality that you want. Then step by step solve your problems, like adding a README.md etc.

user_55160570

I disagree with the other answer. You shouldn't run scripts in __init__.py but in __main__.py instead. 

Projectfolder
├── formatter
│   ├── __init__.py
│   ├── cli
│   │   ├── __init__.py # Import your class module here
│   │   ├── __main__.py # Call your class module here, using __name__ == "__main__"
│   │   ├── your_class_module.py
├── setup.py

If you don't want to supply a readme, just remove that code and enter a description manually.

I use https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-packages instead of manually setting the packages.

You can now install your package by just running pip install ./ like you have been doing before.

After you've done that run: python -m formatter.cli arguments. It runs the __main__.py file you've created in the CLI folder (or whatever you've called it).

An important note about packaging modules is that you need to use relative imports. You'd use from .your_class_module import YourClassModule in that __init__.py for example. If you want to import something from an adjacent folder you need two dots, from ..helpers import HelperClass.

user_87699300

I'm not sure if this is helpful, but usually I package my python scripts using the wheel package:

pip install wheel
python setup.py sdist bdist_wheel

After those two commands a whl package is created in a 'dist' folder which you can then either upload to PyPi and download/install from there, or you can install it offline with the "pip install ${PackageName}.py"

Here's A useful user guide just in case there is something else that I didn't explain:

https://packaging.python.org/tutorials/packaging-projects/

user_132407190
Related