My Notes

Pre-commit

7 min read

Comprehensive Pre-commit Guide

What is Pre-commit?

Pre-commit is a framework for managing and maintaining multi-language pre-commit hooks. It runs checks on your code before each commit, helping catch issues early and maintain code quality standards.

Installation

System Installation

# Python/pip
pip install pre-commit
 
# macOS
brew install pre-commit
 
# Conda
conda install -c conda-forge pre-commit
 
# Ubuntu/Debian
sudo apt install pre-commit

Verify Installation

pre-commit --version

Basic Setup

1. Create Configuration File

Create .pre-commit-config.yaml in your repository root:

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-added-large-files

2. Install Git Hook Scripts

pre-commit install

3. Run Against All Files (Optional)

pre-commit run --all-files

Configuration Deep Dive

Basic Structure

# Optional, set minimum pre-commit version
minimum_pre_commit_version: '2.20.0'
 
# Default language version for hooks
default_language_version:
  python: python3.9
  node: '18.0.0'
 
# Default stages for hooks
default_stages: [commit]
 
# Repository configurations
repos:
  - repo: repository-url
    rev: version-tag
    hooks:
      - id: hook-id
        name: Custom Name
        stages: [commit, push]
        files: '\.py$'
        exclude: '^tests/'
        args: ['--config', 'setup.cfg']

Hook Configuration Options

hooks:
  - id: hook-id
    name: Custom Hook Name           # Override display name
    entry: custom-command           # Override command
    language: python                # Hook language
    files: '\.py$'                 # Files to run on (regex)
    exclude: '^tests/'             # Files to exclude (regex)
    types: [python]                # File types
    types_or: [python, pyi]        # Alternative file types
    exclude_types: [markdown]      # Exclude file types
    args: ['--line-length=88']     # Additional arguments
    stages: [commit, push]         # When to run
    always_run: false              # Run even if no files match
    pass_filenames: true           # Pass filenames to hook
    require_serial: false          # Run in serial vs parallel
    additional_dependencies: []     # Extra packages to install
    minimum_pre_commit_version: '' # Minimum version required

Essential Pre-commit Hooks

Core Hooks

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      # File formatting
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: mixed-line-ending
      
      # Syntax checking
      - id: check-yaml
      - id: check-json
      - id: check-toml
      - id: check-xml
      
      # Security
      - id: check-added-large-files
        args: ['--maxkb=500']
      - id: detect-private-key
      
      # Python specific
      - id: check-ast
      - id: check-builtin-literals
      - id: check-docstring-first
      - id: debug-statements
      - id: requirements-txt-fixer
      
      # Git
      - id: check-merge-conflict
      - id: check-case-conflict

Language-Specific Configurations

Python

repos:
  # Code formatting
  - repo: https://github.com/psf/black
    rev: 23.3.0
    hooks:
      - id: black
        args: [--line-length=88]
 
  # Import sorting
  - repo: https://github.com/pycqa/isort
    rev: 5.12.0
    hooks:
      - id: isort
        args: ["--profile", "black"]
 
  # Linting
  - repo: https://github.com/pycqa/flake8
    rev: 6.0.0
    hooks:
      - id: flake8
        args: [--max-line-length=88, --extend-ignore=E203]
 
  # Type checking
  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.3.0
    hooks:
      - id: mypy
        additional_dependencies: [types-requests]
 
  # Security scanning
  - repo: https://github.com/PyCQA/bandit
    rev: 1.7.5
    hooks:
      - id: bandit
        args: ["-c", "pyproject.toml"]
        exclude: ^tests/
 
  # Documentation
  - repo: https://github.com/pycqa/pydocstyle
    rev: 6.3.0
    hooks:
      - id: pydocstyle

JavaScript/TypeScript

repos:
  # Prettier formatting
  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v3.0.0
    hooks:
      - id: prettier
        types_or: [css, javascript, ts, tsx, json, markdown, yaml]
 
  # ESLint
  - repo: https://github.com/pre-commit/mirrors-eslint
    rev: v8.44.0
    hooks:
      - id: eslint
        files: \.[jt]sx?$
        types: [file]
        additional_dependencies:
          - eslint@8.44.0
          - '@typescript-eslint/parser@5.61.0'
          - '@typescript-eslint/eslint-plugin@5.61.0'
 
  # Package.json sorting
  - repo: https://github.com/pre-commit/mirrors-fixjson
    rev: v4.0.0
    hooks:
      - id: fix-json

Go

repos:
  # Go formatting
  - repo: https://github.com/dnephin/pre-commit-golang
    rev: v0.5.1
    hooks:
      - id: go-fmt
      - id: go-vet-mod
      - id: go-mod-tidy
      - id: go-unit-tests
      - id: golangci-lint

Docker

repos:
  # Dockerfile linting
  - repo: https://github.com/hadolint/hadolint
    rev: v2.12.0
    hooks:
      - id: hadolint-docker

Shell Scripts

repos:
  # Shell script linting
  - repo: https://github.com/shellcheck-py/shellcheck-py
    rev: v0.9.0.5
    hooks:
      - id: shellcheck

Security-Focused Configuration

Secret Detection

repos:
  # Gitleaks
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.24.2
    hooks:
      - id: gitleaks
 
  # GitGuardian
  - repo: https://github.com/gitguardian/ggshield
    rev: v1.25.0
    hooks:
      - id: ggshield
        language: python
        stages: [commit]
 
  # detect-secrets
  - repo: https://github.com/Yelp/detect-secrets
    rev: v1.4.0
    hooks:
      - id: detect-secrets
        args: ['--baseline', '.secrets.baseline']
 
  # TruffleHog
  - repo: https://github.com/trufflesecurity/trufflehog
    rev: v3.45.3
    hooks:
      - id: trufflehog
        name: TruffleHog
        entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail'

Advanced Configurations

Custom Hooks

repos:
  # Local custom hooks
  - repo: local
    hooks:
      - id: custom-script
        name: Run custom validation script
        entry: ./scripts/validate.sh
        language: script
        files: '\.py$'
        
      - id: pytest
        name: pytest
        entry: pytest
        language: python
        pass_filenames: false
        always_run: true
        
      - id: custom-python
        name: Custom Python Check
        entry: python scripts/custom_check.py
        language: python
        files: '\.py$'

Conditional Hooks

repos:
  - repo: https://github.com/psf/black
    rev: 23.3.0
    hooks:
      - id: black
        # Only run on Python files in src/ directory
        files: '^src/.*\.py$'
        
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: check-yaml
        # Exclude specific files
        exclude: '^(docker-compose|k8s).*\.ya?ml$'

Multi-Stage Configuration

repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: trailing-whitespace
        stages: [commit]
      - id: check-added-large-files
        stages: [commit, push]
        
  - repo: https://github.com/psf/black
    rev: 23.3.0
    hooks:
      - id: black
        stages: [commit]

Command Reference

Installation and Setup

# Install pre-commit hooks
pre-commit install
 
# Install for different hook types
pre-commit install --hook-type pre-commit
pre-commit install --hook-type pre-push
pre-commit install --hook-type commit-msg
 
# Install all hook types
pre-commit install --install-hooks

Running Hooks

# Run all hooks on staged files
pre-commit run
 
# Run all hooks on all files
pre-commit run --all-files
 
# Run specific hook
pre-commit run black
 
# Run hooks on specific files
pre-commit run --files file1.py file2.py
 
# Run hooks for specific stage
pre-commit run --hook-stage push

Maintenance

# Update hooks to latest versions
pre-commit autoupdate
 
# Update specific repository
pre-commit autoupdate --repo https://github.com/psf/black
 
# Clean cache
pre-commit clean
 
# Uninstall hooks
pre-commit uninstall
 
# Sample configuration
pre-commit sample-config

Debugging

# Verbose output
pre-commit run --verbose
 
# Show diff for failing hooks
pre-commit run --show-diff-on-failure
 
# Run hooks without modifying files
pre-commit run --no-stash

CI/CD Integration

GitHub Actions

name: pre-commit
on:
  pull_request:
  push:
    branches: [main]
 
jobs:
  pre-commit:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-python@v4
      with:
        python-version: 3.x
    - uses: pre-commit/action@v3.0.0

GitLab CI

pre-commit:
  stage: test
  image: python:3.9
  before_script:
    - pip install pre-commit
  script:
    - pre-commit run --all-files

Jenkins

pipeline {
    agent any
    stages {
        stage('Pre-commit') {
            steps {
                sh 'pip install pre-commit'
                sh 'pre-commit run --all-files'
            }
        }
    }
}

Performance Optimization

Parallel Execution

# Default - hooks run in parallel when possible
default_stages: [commit]
 
# Force serial execution for specific hooks
repos:
  - repo: local
    hooks:
      - id: slow-hook
        name: Slow Hook
        entry: ./slow-script.sh
        language: script
        require_serial: true

Selective File Processing

repos:
  - repo: https://github.com/psf/black
    rev: 23.3.0
    hooks:
      - id: black
        # Only run on modified Python files
        files: '\.py$'
        # Exclude test files for faster commits
        exclude: '^tests/'

Caching

# Pre-commit automatically caches hook environments
# Cache location: ~/.cache/pre-commit/
 
# Clean cache if needed
pre-commit clean
 
# Environment variables for cache control
export PRE_COMMIT_HOME=/custom/cache/path

Troubleshooting

Common Issues

Hook fails to install:

# Clear cache and reinstall
pre-commit clean
pre-commit install --install-hooks

Hooks not running:

# Check installation
ls -la .git/hooks/
cat .git/hooks/pre-commit
 
# Reinstall if needed
pre-commit install

Permission errors:

# Make scripts executable
chmod +x scripts/custom-hook.sh
 
# Or specify in config
- id: custom-hook
  entry: bash scripts/custom-hook.sh
  language: script

Python import errors:

# Add dependencies to hook
- id: mypy
  additional_dependencies: [types-requests, types-PyYAML]

Skip Hooks Temporarily

# Skip all hooks for one commit
git commit --no-verify -m "Emergency fix"
 
# Skip specific hook
SKIP=flake8 git commit -m "Skip flake8"
 
# Skip multiple hooks
SKIP=flake8,mypy git commit -m "Skip linting"

Best Practices

Repository Structure

project/
├── .pre-commit-config.yaml
├── .pre-commit-hooks.yaml     # If creating custom hooks
├── scripts/
│   ├── pre-commit-setup.sh
│   └── custom-validators/
├── .gitignore
└── README.md

Configuration Guidelines

  1. Start Simple: Begin with basic hooks and gradually add more
  2. Fast Feedback: Prioritize fast hooks for better developer experience
  3. Fail Fast: Put syntax checkers before formatters
  4. Consistent Versions: Pin hook versions for reproducible builds
  5. Document Exceptions: Clearly document why certain files are excluded

Sample Complete Configuration

# .pre-commit-config.yaml
minimum_pre_commit_version: '2.20.0'
 
default_language_version:
  python: python3.9
 
repos:
  # Basic file checks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.4.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-json
      - id: check-added-large-files
      - id: detect-private-key
      - id: check-merge-conflict
 
  # Python
  - repo: https://github.com/psf/black
    rev: 23.3.0
    hooks:
      - id: black
 
  - repo: https://github.com/pycqa/isort
    rev: 5.12.0
    hooks:
      - id: isort
        args: ["--profile", "black"]
 
  - repo: https://github.com/pycqa/flake8
    rev: 6.0.0
    hooks:
      - id: flake8
 
  # Security
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.24.2
    hooks:
      - id: gitleaks
 
  # Documentation
  - repo: https://github.com/pre-commit/mirrors-prettier
    rev: v3.0.0
    hooks:
      - id: prettier
        types_or: [markdown, yaml]

Team Adoption Strategy

  1. Gradual Implementation: Start with non-disruptive hooks
  2. Team Training: Educate team on pre-commit benefits
  3. Clear Documentation: Provide setup and troubleshooting guides
  4. Consistent Environment: Use same Python/Node versions across team
  5. Escape Hatches: Document when and how to skip hooks

This comprehensive guide should cover most pre-commit use cases and help you implement a robust code quality pipeline for your projects.

Graph View