Conditionally ignore files in git

Per man gitignore there are four sources of patterns for ignoring files. Command-line arguments are probably too much hassle; .gitignore is itself version-controlled (unless you include .gitignore in it), which creates complications. That leaves $GIT_DIR/info/exclude and the file listed in config variable core.excludesFile. Sadly, it seems that $GIT_DIR/info/exclude has to be a file and not a directory whose contents are concatenated.

A somewhat complicated solution which probably doesn't conflict directly with anything you're already doing would be:

1. Create a script which updates $GIT_DIR/info/exclude to list the .pdf files for which there are .tex files. If you want to be really cautious you could delimit a section of the file with comments and replace only that section.
2. alias git to invoke the script and then pass the arguments along to /usr/bin/git. (In practice this probably means making the script itself pass the arguments along, and make the script the alias).

posted almost 3 years ago

CC BY-SA 4.0

Peter Taylor‭

1302 reputation 7 34 141 7

Something like automatically ignoring all .pdf files for which a .tex of the same name exists?

We can do something close to that. We can reject the commit if your change list contains pdf and tex files with the same name and path with a git hook. You can then remove the files and try again. This will only work for your repository, your collaborators will have to also put this hook in their repository to get the same protection.

Create a file in your .git/hooks folder with the name pre-commit.

#!/bin/sh

# This is just to provide a way to bypass just this check.
if test -n "${SKIP_PDF_CHECK+1}"
then
    echo "Check for PDF generated files in commit skipped."
    exit 0
fi

# This command creates a table of file name and the number of times the name appears in the output of git diff.
# Example input / output:
#   Given git diff shows these changed files:
#        lonely.pdf
#        other.pdf
#        singular.tex
#        oops.pdf
#        oops.tex
#        oops.odd.periods.pdf
#        oops.odd.periods.tex
#   counts will be:
#        1 lonely
#        1 other
#        1 singular
#        2 oops
#        2 oops.odd.periods
# 
# Comments for each command:
#   Get the list of changes including Added Copied Modified. See other options: https://git-scm.com/docs/git-diff#Documentation/git-diff.txt---diff-filterACDMRTUXB82308203
#   Filter down to just the files that end in pdf or tex
#   Remove the extension.
#   Sort the list in preparation for uniq
#   Show only unique lines and include the count
counts=$(git diff --cached --name-only --diff-filter=ACM | awk '/\.pdf|\.tex$/ {print}' | awk -F. 'BEGIN { OFS = FS }; NF { NF -= 1 }; {print $0};' | sort | uniq -c)

# Print the output of $counts
# Check if any of the counts are more than 1. Which would mean there are 2 files with the same name but different extension. 
if echo "$counts" | awk '$1 > 1 {exit 1}'
then
    echo "No generated files detected in commit."

    # Allow git commit.
    exit 0
else
    echo "The following file names look generated. Remove them or skip this check with SKIP_PDF_CHECK=1 git commit";
    # Print the offending files.
    echo "$counts" | awk '$1 > 1 {print $2}'

    # Prevent git commit.
    exit 1
fi

This script will now run every time you do git commit. If you want to skip this check for whatever reason you can do SKIP_PDF_CHECK=1 git commit.

You can do pretty much any algorithm for detecting generated files since the hook is just a shell script. I will also caveat that script hasn't been tested thoroughly. It may not work with things like emojiis in filenames and such.

Of course, it's strange that these files are getting accidentally added to the change list at all. Are you using git add . or similar? Perhaps use git add -ip to have git interactively prompt you about each change so you can review them.

posted almost 3 years ago

CC BY-SA 4.0

3y ago

jmathew‭

275 reputation 0 6 25 9

I'm not familiar with Latex, but it seems the PDFs are generated from the Latex files.

It then seems the real problem is that you are trying to keep source and objects derived from that source in the GIT repository. Ideally, a GIT repository is only for the actual source files (those directly edited by humans). Put the files that are automatically derived from source elsewhere. This can be automated with your build scripts putting derived objects in a different place. That different place could be a subdirectory within the repository that is added to the .gitignore list.

Another possibility is to have a cleaner script that you run before each commit. This script would delete all the derived objects.

Yet another possibility, although I really don't like this one, is to have the cleaner script instead edit the .gitignore file to ignore the known derived objects, based on the existence of particular source files.

posted almost 3 years ago

CC BY-SA 4.0

Olin Lathrop‭

1069 reputation 1 24 90 11