Stream: hack-projects

Topic: cookiecutter-template

view this post on Zulip Anderson Banihirwe (May 28 2020 at 22:03):

:megaphone:

Hi @all ,

For anyone who is interested in creating a Python project, but wants to avoid the boilerplate of a Python project ; I put together a minimal, cookiecutter template (with a well-defined, standard project structure).

The cookiecutter template resides here.
See https://github.com/NCAR/pypackage-template for more details.

Quick Start

conda install  -c conda-forge cookiecutter

Or 

python -m pip install cookiecutter

cookiecutter https://github.com/NCAR/cookiecutter-pypackage.git

Example

❯ cookiecutter https://github.com/NCAR/pypackage-template                            (base) 15:49:38
maintainer_full_name [Foo Bar]: Homer Simpson
maintainer_email [foo@ucar.edu]: homer@springfield.io
github_username_or_organization [NCAR]: SNPP
project_name [Python Boilerplate]: Time Traveling Toaster
project_slug [time_traveling_toaster]:
project_short_description [Python Boilerplate contains all the boilerplate you need to create a Python package.]: A Toaster to transport you to prehistoric times: be careful not to change anything if you travel back in time
Select open_source_license:
1 - MIT license
2 - BSD license
3 - ISC license
4 - Apache Software License 2.0
5 - GNU General Public License v3
Choose from 1, 2, 3, 4, 5 [1]: 2

~/devel/personal via C base via  96%/36% took 45s
❯ cd time_traveling_toaster/                                                         (base) 15:50:28

devel/personal/time_traveling_toaster via 🐍 v3.7.6 via C base via  96%/36%
❯ tree -L 3                                                                          (base) 15:50:41
.
├── LICENSE
├── MANIFEST.in
├── README.rst
├── ci
│   └── environment.yml
├── codecov.yml
├── pre-commit-config.yaml
├── requirements.txt
├── setup.cfg
├── setup.py
├── tests
│   ├── __init__.py
│   └── test_sample.py
└── time_traveling_toaster
    └── __init__.py

3 directories, 12 files

devel/personal/time_traveling_toaster via 🐍 v3.7.6 via C base via  95%/36%
❯ cat setup.py
setup(
    maintainer="Homer Simpson",
    maintainer_email='homer@springfield.io',
    python_requires='>=3.6',
    classifiers=[
        'Development Status :: 2 - Pre-Alpha',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: BSD License',
        'Natural Language :: English',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: 3.8',
        'Topic :: Scientific/Engineering',
        'Operating System :: OS Independent',
        'Intended Audience :: Science/Research',
    ],
    description='A Toaster to transport you to prehistoric times: be careful not to change anything if you travel back in time',
    install_requires=requirements,
    license='BSD license',
...

view this post on Zulip Keith Lindsay (Jun 05 2020 at 21:24):

@Anderson Banihirwe , in the pypackage-template repo, it looks like you've put arguments for black into the file .pre-commit-config.yaml, while the arguments for flake8 are in the file setup.cfg. What's the rationale for putting these tool's arguments in different files?

view this post on Zulip Michael Levy (Jun 05 2020 at 22:00):

@Anderson Banihirwe a couple of comments about the pre-commit setup:

  1. In python 3.8, the pre-commit checks failed with
An unexpected error has occurred: CalledProcessError: command: ('/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/bin/python', '-mpip', 'install', '.')
return code: 1
expected return code: 0
stdout: (none)
stderr:
    Traceback (most recent call last):
      File "/glade/work/mlevy/miniconda3/envs/python_checks/lib/python3.8/runpy.py", line 194, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/glade/work/mlevy/miniconda3/envs/python_checks/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/__main__.py", line 23, in <module>
        from pip._internal.cli.main import main as _main  # isort:skip # noqa
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 10, in <module>
        from pip._internal.cli.autocompletion import autocomplete
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
        from pip._internal.cli.main_parser import create_main_parser
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
        from pip._internal.cli import cmdoptions
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 24, in <module>
        from pip._internal.cli.progress_bars import BAR_TYPES
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/cli/progress_bars.py", line 12, in <module>
        from pip._internal.utils.logging import get_indentation
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/utils/logging.py", line 18, in <module>
        from pip._internal.utils.misc import ensure_dir
      File "/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 20, in <module>
        from pip._vendor import pkg_resources
    ImportError: cannot import name 'pkg_resources' from 'pip._vendor' (/glade/u/home/mlevy/.cache/pre-commit/repopeptvfuh/py_env-python3.8/lib/python3.8/site-packages/pip/_vendor/__init__.py)

but everything seemed to work after downgrading to 3.7

  1. There are checks in the pre-commit setup that are not in the continuous integration, which seems like an odd choice. Code that passed all the CI tests failed in pre-commit:
$ git commit
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check docstring is first.................................................Passed
Check Yaml...........................................(no files to check)Skipped
Fix double quoted strings................................................Failed
- hook id: double-quote-string-fixer
- exit code: 1
- files were modified by this hook

Fixing strings in cesmcatalog/gen_CESM_catalog.py

black....................................................................Passed
flake8...................................................................Passed
seed isort known_third_party.............................................Failed
- hook id: seed-isort-config
- exit code: 1
- files were modified by this hook
isort....................................................................Passed

Is there a reason why those first five checks are in pre-commit but not in the continuous integration? It seems like you should be able to include the formatting checks via black or flake8. Also, why is isort flagging an error here but not on github?

view this post on Zulip Anderson Banihirwe (Jun 07 2020 at 11:11):

Anderson Banihirwe , in the pypackage-template repo, it looks like you've put arguments for black into the file .pre-commit-config.yaml, while the arguments for flake8 are in the file setup.cfg. What's the rationale for putting these tool's arguments in different files?

Black doesn't (has refused to) support the setup.cfg config file. If interested, see this issue for more details. Black supports pyproject.toml configuration file though. However, I didn't want to have an additional configuration file for just black. Also, a while ago I ran into some issues when I tried using pyproject.toml and setup.cfg together.

Note: If you've never heard of pyproject.toml and want to go down a rabbit hole :grinning:, here are two great blog posts on it:

view this post on Zulip Anderson Banihirwe (Jun 07 2020 at 11:20):

@Anderson Banihirwe a couple of comments about the pre-commit setup:
In python 3.8, the pre-commit checks failed with

This issue appears to be the same issue described by @Riley Brady here

but everything seemed to work after downgrading to 3.7

Riley was able to get it to work without needing to downgrade to Python 3.7 as he described in his reply:

Hm I changed out the repo for pre-commit on cheyenne and it didn't work. Switched to an interactive node on casper and it worked. Not really sure what's going on here.

It would be nice to know what's going on exactly

view this post on Zulip Anderson Banihirwe (Jun 07 2020 at 11:29):

Is there a reason why those first five checks are in pre-commit but not in the continuous integration? It seems like you should be able to include the formatting checks

There's nothing stopping us from having them in the continuous integration. The short answer is that it wasn't straightforward to add them to the continuous integration, and I was lazy :grinning:

When I get a chance, I will look into it and will update the cookiecutter template...

Last updated: May 16 2025 at 17:14 UTC