pre-commit is a tool to automatically run software tools, referred to as
hooks, when committing code to a versioning control repository like
See the pre-commit website.
To enable this:
Install pre-commit, for instance
conda install -y pre-commit.
The following pre-commit hooks are used:
An opinionated autoformatter.
Checks xml files for proper format.
Checks yaml files for proper format.
Format files with ClangFormat (optional).
A style checker with many different plugins to enforce different rules.
Feed all XML files through xmllint.
An opinionated import sorter.
Performs type checking on the code (optional).
An extremely fast Python linter, written in Rust.
ts_pre_commit_conf is a tool to generate configuration files for the
pre-commit hooks for the Telescope and Site SoftWare (TSSW) team at the Vera C. Rubin Observatory.
These configuration files facilitate
pre-commit to maintain
flake8 compliance and
isort imports sorting.
mypy is used, a configuration file can be generated for that as well.
The configuration files are generated by using the
Since TSSW uses
mamba) for their software installations, a conda recipe named
ts-pre-commit-config was created for this project and the conda package was made available in the
lsstts conda channel.
To install it, run
conda install -y -c lsstts ts-pre-commit-config.
In order for the
generate_pre_commit_conf command to know which
pre-commit hooks need to be applied, a
.ts-pre-commit-config.yaml configuration file needs to be created.
.ts-pre-commit-config.yaml file contains information that indicates for every
pre-commit whether it should be applied or not.
.ts-pre-commit-config.yaml configuration file can be generated using the
generate_pre_commit_conf --create command, or manually.
For more information on the command line options, use
An example where all
pre-commit hooks are enabled is:
generate_pre_commit_conf command fails with a comprehensive error message if a mandatory or optional
pre-commit hook is missing.
isort hooks are mandatory for TSSW projects, so those hooks need to be set to
Setting one or more of the mandatory hooks to
false will make the
generate_pre_commit_conf command fail with a comprehensive error message.
towncrier hooks are optional and do not get included by default.
They can be included by using the corresponding
--with-XXX command line option.
mypy hook is optional and gets included by default.
It can be excluded by using the corresponding
--no-mypy command line option.
Setting one or more of the optional hooks to
false or omitting them from the
.ts-pre-commit-config.yaml file will make the
generate_pre_commit_conf command skip those hooks.
It is possible to add custom
pre-commit hook configuration files.
This may be necessary in case the TSSW configuration doesn’t match what is required for the project.
In such a case, the configuration file needs to be committed to git.
By default, the
generate_pre_commit_conf command skips overwriting existing hook configuration files.
If the hook configuration files need to be overwritten, for instance when the TSSW configuration rules change, then the
--overwrite command line option can be used.
The configuration files will be updated whenever the pre-commit hooks get updated.
Apart from that, all generated configuration file names get added to
.gitignore as well.
The TSSW Jenkins CI jobs update these configuration files and execute pre-commit every time the jobs run, so you know that you need to update them locally and fix your code if a job fails.
You can update the config file locally by running the
generate_pre_commit_conf command again.
Projects with a
There are several situations in which a project already may have a
One is when another developer has configured the project and you have cloned the repo.
Another is when a project was configured by you and one or more
pre-commit hooks were updated.
A third is when the project previously didn’t use
mypy and now it does.
In all cases the config files for
pre-commit and the hooks can easily be created or updated.
All that needs to be done is to run the
generate_pre_commit_conf command without any arguments.
generate_pre_commit_conf command will read the existing
.ts_pre_commit_config.yaml file and use that to do its work.
Note that this will overwrite the existing
.pre-commit-config.yaml files as well as any existing config file for the
Projects without a
For a project that already has a
.pre-commit-config.yaml configuration file, execute these steps:
.pre-commit-config.yamlfile to see if the project uses clang-format and/or mypy or not.
git rm --cached .pre-commit-config.yaml.
setup.cfgif it exists with
git rm setup.cfg(note: this file has been replaced by
pytest-asyncio, if present. See Conda for information on what
pyproject.tomlto remove all linter configuration. This includes all
flake8options and the
addoptsline in the
tool.pytest.ini_optionssection. See Python for information on what
Then, in all cases, execute these steps:
ts_pre_commit_confif not already done, as per the installation instructions above.
generate_pre_commit_conf --createif clang-format and mypy are used. Use the
--no-clang-formatoption to exclude clang-format and the
--no-mypyoption to exclude mypy.
Add the newly created
.ts_pre_commit_config.yamlto git with
git add .ts_pre_commit_config.yaml.
Run the pre-commit hooks on all of your code, using
pre-commit run --all-files. If this changes anything, fix as needed:
Fix mypy errors.
If isort changes any
__init__.pyfiles, run unit tests and fix any breakage. Other isort changes should be innocuous, but it never hurts to run unit tests.
Changes made by black should never break anything.
Once this is all done, create a git commit to reflect the change with
git commit -a -m "Use ts_pre_commit_conf.".
Adding a new hook#
In order to add a new hook, do the following:
Create a new ticket branch in the
ts_pre_commit_confproject following the development-workflow.
Add a new entry to the
registrydict providing the following information:
pre_commit_config: the config for the
.pre-commit-config.yamlfile. Provide this as a triple quoted string without leading or trailing whitespace apart from a newline character at the end. See the other hooks in the registry for examples.
config_file_name: the name of the config file of the hook, or None if the hook doesn’t require a config file.
config: the config file contents as a string. Provide this as a triple quoted string without leading or trailing whitespace apart from a newline character at the end. See the other hooks in the registry for examples. Note that this needs to be set to None if config_file_name is set to None.
rule_type: this can be one of
MANDATORY: The hook is mandatory for all TSSW projects. Adding such a type of hook needs to be discussed at TSSW standup first.
OPT_IN: The hook is optional and does not get included by default. The majority of the hooks will be of the OPT_IN rule type.
OPT_OUT: The hook is optional and gets included by default. Adding such a type of hook needs to be discussed at TSSW standup first.
config may be omitted when they are
excludable when they are
None is the default value.