Pre-Commit Validation

Let me describe a scenario for you, and you tell me if this has happened to you or not. You are working on a feature-branch, and your code is working as expected. You add and commit your files to the local repository and then push it to the remote repo. A CI/CD pipeline kicks in, and it goes through the steps of validating your changes against the repo’s set rules (code formatting, testing, etc.). Two minutes have passed, and you notice that your code has failed the validation because the formatting was incorrect. You quickly apply the formatting and commit again. Now, five minutes have passed, and you find out that the code testing is failing. Well, you just lost seven minutes, and the pattern of inefficiencies continues.

Usually, the repository has some sort of local testing capabilities which each developer should run before pushing their code to the repo. But we are all human, we make mistakes, we forget, we rush, etc. What if there was a way to enforce local checking before allowing the commit to happen? Guess what!? Such technology exists, and I am going to show how it works.

Git Hooks

Git hooks are scripts which can be run automatically for certain events that occur in the Git repository. There are two types of Git hooks:

1. client-side hooks - Executed for the events on the local repository. For example, git commit or git merge.
2. server-side hooks - Executed in the remote repository. For example, git push.

The table below contains a list of supported git hooks:

Hook	Event	Description
pre-commit	git commit	Does not take any inputs and runs the specified code before the commit. The code must exit with status code of 0 or the commit will fail.
pre-merge-commit	git merge	Does not take any inputs and runs the specified code after the merge. The code must exit with status code of 0 or the merge will fail.
commit-msg	git commit / git merge	It takes the name of the file that holds the proposed commit log message as an input. Exiting with a non-zero status causes the command to abort.
post-commit	git commit	Does not take any inputs and runs the specified code after the commit. This hook cannot affect the outcome of the git commit.

A full list of supported hooks and detailed explanation for each hook can be found here. Also, this git repository contains some examples for several of the hooks mentioned in the table above.

Creating a Git Hook

To create a git hook, all you have to do is create an executable file in the .git/hooks folder with an appropriate name (e.g., post-merge) with the desired logic.

NOTE
The repository should already contain the .git/hooks folder with some examples.

$ tree -a
.
├── .git
│   ├── branches
│   ├── COMMIT_EDITMSG
│   ├── config
│   ├── description
│   ├── HEAD
│   ├── hooks
│   │   ├── applypatch-msg.sample
│   │   ├── commit-msg.sample
│   │   ├── post-update.sample
│   │   ├── pre-applypatch.sample
│   │   ├── pre-commit.sample
│   │   ├── prepare-commit-msg.sample
│   │   ├── pre-push.sample
│   │   ├── pre-rebase.sample
│   │   └── update.sample

pre-commit Python Package

In this blog, I will cover how to execute validation on the modified code before allowing git to commit it locally. For this, I could have easily created a pre-commit hook with desired logic and called it a day, but that is not fun and not very scalable. Instead, I will use a pre-commit python package which was developed to ease the hook creation process and includes much more functionality.

The pre-commit python package can be installed as follows:

$ python -m pip install pre-commit

After the installation, check the version installed to validate successful installation and operation of the package.

$ pre-commit --version
pre-commit 2.17.0

Now let’s take a look at the repository in which we will be implementing the pre-commit hook. First we will need to clone it from GitHub.

$ git clone git@github.com:TachTech-Engineering/pre-commit-validation.git
Cloning into 'pre-commit-validation'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (9/9), done.
Checking connectivity... done.

Next, let’s check the contents of the repo.

$ cd pre-commit-validation/
$ ls -Al
total 20
-rw-r--r--  1 user user   46 Sep  8 08:43 awesome_python_app.py
drwxr-xr-x  8 user user 4096 Sep  8 08:43 .git
-rw-r--r--  1 user user   86 Sep  8 08:43 README.md

As you can see the repo has very few files in it, but it is more than enough for our use case. Now let’s check the contents of the .git/hooks folder to make sure there are no existing hooks.

$ ls -Al .git/hooks/
total 48
-rwxr-xr-x 1 user user  478 Sep  8 08:43 applypatch-msg.sample
-rwxr-xr-x 1 user user  896 Sep  8 08:43 commit-msg.sample
-rwxr-xr-x 1 user user  189 Sep  8 08:43 post-update.sample
-rwxr-xr-x 1 user user  424 Sep  8 08:43 pre-applypatch.sample
-rwxr-xr-x 1 user user 1642 Sep  8 08:43 pre-commit.sample
-rwxr-xr-x 1 user user 1239 Sep  8 08:43 prepare-commit-msg.sample
-rwxr-xr-x 1 user user 1348 Sep  8 08:43 pre-push.sample
-rwxr-xr-x 1 user user 4898 Sep  8 08:43 pre-rebase.sample
-rwxr-xr-x 1 user user 3610 Sep  8 08:43 update.sample

Great, now use the pre-commit install command to create the pre-commit git hook in the appropriate folder.

NOTE
You can use the --hook-type command argument to specify a different git hook type to be created. For example, $ pre-commit install --hook-type post-commit

$ pre-commit install
pre-commit installed at .git/hooks/pre-commit
$ ls -Al .git/hooks/
total 52
-rwxr-xr-x 1 user user  478 Sep  8 08:43 applypatch-msg.sample
-rwxr-xr-x 1 user user  896 Sep  8 08:43 commit-msg.sample
-rwxr-xr-x 1 user user  189 Sep  8 08:43 post-update.sample
-rwxr-xr-x 1 user user  424 Sep  8 08:43 pre-applypatch.sample
-rwxr-xr-x 1 user user  634 Sep  8 08:54 pre-commit   <- File which was created by the previous command.
-rwxr-xr-x 1 user user 1642 Sep  8 08:43 pre-commit.sample
-rwxr-xr-x 1 user user 1239 Sep  8 08:43 prepare-commit-msg.sample
-rwxr-xr-x 1 user user 1348 Sep  8 08:43 pre-push.sample
-rwxr-xr-x 1 user user 4898 Sep  8 08:43 pre-rebase.sample
-rwxr-xr-x 1 user user 3610 Sep  8 08:43 update.sample

We are ready to create the .pre-commit-config.yaml file which will be used for the pre-commit validation.

NOTE
You can use the pre-commit sample-config command to generate a very basic configuration file.

The contents of the file should be as follows:

$ cat .pre-commit-config.yaml
repos:
  - repo: https://github.com/psf/black
    rev: 22.1.0
    hooks:
      - id: black
        name: Black Formatting (Check Only)
        args: [
          ., --diff, --check
        ]
        verbose: true
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.1.0
    hooks:
      - id: trailing-whitespace
        name: Trim Trailing Whitespace
        verbose: true
      - id: end-of-file-fixer
        name: End With Newline
        verbose: true

So what the pre-commit will do when the git commit command is issued is it will go through each repo and execute the defined hook(s). Our .pre-commit-config.yaml will download the code from the https://github.com/psf/black repo and run the black command with . --diff --check arguments, then it will download the next repo and execute the trailing-whitespace and end-of-file-fixer hooks. A question might arise, “Why do those repositories work with the pre-commit?” The answer is simple: they have a special file called .pre-commit-hooks.yaml which allows the pre-commit to read and understand the hooks set in our config.

The explanation of the keys in the file can be found in the below table:

Key	Description
repo	The repository URL to git clone from
rev	The revision or tag to clone at
hooks	The hook mapping configuration
name	(optional) Overrides the name of the hook
args	(optional) List of additional arguments to pass to the hook.
verbose	(optional) Prints the output even when the hook passes.

The full list of all available keys and their descriptions can be found here.

Great, let’s continue by testing how it works. To run all the hooks use pre-commit run -a

$ pre-commit run -a
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/psf/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Black Formatting (Check Only)............................................Failed
- hook id: black
- exit code: 1

--- awesome_python_app.py       2022-09-26 18:11:05.882461 +0000
+++ awesome_python_app.py       2022-09-26 20:31:27.260010 +0000
@@ -1,5 +1,5 @@
 import sys


 def main():
-  print(sys.version)
+    print(sys.version)
would reformat awesome_python_app.py

Oh no! 💥 💔 💥
1 file would be reformatted.

Trim Trailing Whitespace.................................................Passed
End With Newline.........................................................Passed

Now, let’s see the same thing but when the git commit command is issued. For that, create an empty file, add it, and then commit it to the local repository to see the result.

$ touch new_feature.py
$ git add new_feature.py
$ git commit -m "New feature"
Black Formatting (Check Only)............................................Failed
- hook id: black
- exit code: 1

--- awesome_python_app.py       2022-09-26 18:11:05.882461 +0000
+++ awesome_python_app.py       2022-09-26 20:41:20.224781 +0000
@@ -1,5 +1,5 @@
 import sys


 def main():
-  print(sys.version)
+    print(sys.version)
would reformat awesome_python_app.py

Oh no! 💥 💔 💥
1 file would be reformatted, 2 files would be left unchanged.

Trim Trailing Whitespace.................................................Passed
End With Newline.........................................................Passed

If you check the status of the repository, you will see that our new file was not committed.

$ git status
On branch main
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        new file:   new_feature.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        .pre-commit-config.yaml

What if you need to run more sophisticated checks like python tests? Well that also can be done with the pre_commit python package. To see this in action, change the branch to pytest_check using the git checkout pytest_check command and list the available files:

$ git checkout pytest_check
Switched to branch 'pytest_check'
$ ls -Al
total 36
-rw-r--r--  1 user user   57 Sep  8 10:39 awesome_python_app.py
drwxr-xr-x  8 user user 4096 Sep  8 10:39 .git
-rw-r--r--  1 user user  641 Sep  8 10:39 .pre-commit-config.yaml
-rw-r--r--  1 user user   86 Sep  8 08:43 README.md
-rw-r--r--  1 user user  266 Sep  8 10:39 test_awesome_python_app.py

This branch contains one more file that is used to test our awesome_python_app.py using Pytest. Now let’s take a look at the .pre-commit-config.yaml file since it has some changes to it.

$ cat .pre-commit-config.yaml
repos:
  - repo: https://github.com/psf/black
    rev: 22.1.0
    hooks:
      - id: black
        name: Black Formatting (Check Only)
        args: [
          ., --diff, --check
        ]
        verbose: true
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.1.0
    hooks:
      - id: trailing-whitespace
        name: Trim Trailing Whitespace
        verbose: true
      - id: end-of-file-fixer
        name: End With Newline
        verbose: true
  - repo: local
    hooks:
      - id: pytest
        name: Running PyTest
        entry: pytest
        args: [-v]
        pass_filenames: false
        language: system
        always_run: true
        verbose: true

As you can see, we added a new repo which references the local repository. Some additional keys are also present, and the description for them can be found below:

• entry - The executable to run.
• language - The language of the hook - tells pre-commit how to install the hook.
• always_run - If true, this hook will run even if there are no matching files.
• pass_filenames - if false, no filenames will be passed to the hook.

Let’s try to commit one more time and see how it works.

$ git status
On branch pytest_check
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        new file:   new_feature.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        __pycache__/
$ git commit -m "Run PyTest before commit"
Black Formatting (Check Only)............................................Failed
- hook id: black
- duration: 0.23s
- exit code: 1

--- awesome_python_app.py       2022-03-15 17:39:05.347626 +0000
+++ awesome_python_app.py       2022-03-15 18:02:31.348056 +0000
@@ -1,6 +1,6 @@
 import sys


 def main():
-  print(sys.version)
-  return 1
+    print(sys.version)
+    return 1
would reformat awesome_python_app.py
--- test_awesome_python_app.py  2022-03-15 17:39:05.347626 +0000
+++ test_awesome_python_app.py  2022-03-15 18:02:31.358042 +0000
@@ -1,13 +1,13 @@
 import awesome_python_app

+
 def test_main_fail():
-    """Checks main function.
-    """
+    """Checks main function."""
     value = 2
     assert value == awesome_python_app.main()

+
 def test_main_pass():
-    """Checks main function.
-    """
+    """Checks main function."""
     value = 1
     assert value == awesome_python_app.main()
would reformat test_awesome_python_app.py

Oh no! 💥 💔 💥
2 files would be reformatted, 1 file would be left unchanged.

Trim Trailing Whitespace.................................................Passed
- hook id: trailing-whitespace
- duration: 0.04s
End With Newline.........................................................Passed
- hook id: end-of-file-fixer
- duration: 0.04s
Running PyTest...........................................................Failed
- hook id: pytest
- duration: 0.58s
- exit code: 1

============================= test session starts ==============================
platform linux -- Python 3.8.9, pytest-6.1.2, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python
cachedir: .pytest_cache
rootdir: /home/armartirosyan/Projects/pre-commit-validation
plugins: f5-sdk-3.0.21
collected 2 items

test_awesome_python_app.py::test_main_fail FAILED                        [ 50%]
test_awesome_python_app.py::test_main_pass PASSED                        [100%]

=================================== FAILURES ===================================
________________________________ test_main_fail ________________________________

    def test_main_fail():
        """Checks main function.
        """
        value = 2
>       assert value == awesome_python_app.main()
E       assert 2 == 1
E         +2
E         -1

test_awesome_python_app.py:7: AssertionError
----------------------------- Captured stdout call -----------------------------
3.8.9 (default, Apr  3 2021, 01:02:10)
[GCC 5.4.0 20160609]
=========================== short test summary info ============================
FAILED test_awesome_python_app.py::test_main_fail - assert 2 == 1
========================= 1 failed, 1 passed in 0.12s ==========================

Conclusion

Git hooks are very powerful tools that can be used to enforce certain policies on the developers before allowing them to commit changes to the local repository. That being said, the hooks have a minor limitation, they are not copied with the repository when cloned. That is why the pre-commit python package has a requirement for each developer to run the pre-commit install command after cloning the repository. It is a minor inconvenience for greater good.

Armen
GitHub: @armartirosyan
Twitter: @armartirosyan

TachTech Engineering can help make your DevOps initiatives successful and align to best practices. Our sales team can help connect you to the TachTech Engineering team.

Share this on:

• Twitter
• LinkedIn
• Facebook