Unrelated to Python, but is anyone aware of a tool like Make that can handle file path wildcards in prerequisites?
For example, let's say I have a rule that consumes *.json to produce some target. I want it to rebuild that target if any *.json files are added or removed (or modified).
As far as I'm aware, Make relies on file system timestamps, so even if it supported wildcards (I'm not sure if it does or not), it wouldn't be able to notice that foo.json has been deleted and a rebuild of the target is needed.
I thought I'd ask here before I go build such a tool (to support incremental rebuilds of a static site).
Edit to add that my particular use case has a non-prescriptive set of directories, nested a few levels deep. I'm actually realizing that that is probably a bigger hurdle for using something like Make (e.g. transform all *.md files, no matter how deep; but copy everything else, like images, verbatim; oh, and also then aggregate all *.md files into an Atom feed). Yes, I know this is asking a lot of something like Make!
I did spend quite a bit of time researching approaches, but I didn't come across this particular idea. I'll give it a try. Thanks!
Note: in my case, I have a directory structure that's a few levels deep, with an non-prescriptive set of directories (one subdirectory per category, with no limit on the set of categories). Maybe Make handles directories better than I realized (I'd always seen it recommended to use a Makefile in each directory--something I want to avoid).
I have used this method (directory mod-time triggering, let's say) for a simulation-summarizer which analyzes whatever pile of simulation-output-files happen to be in a given directory. If you run a new simulation, the directory changes and the analysis is re-done by running "make".
I used the Gnu make $(wildcard ...) for the template expansion, instead of using shell expansion. This is to take care of the no-file case, so that jsons/*.json will expand to nothing rather than to the literal jsons/*.json (which does not exist).
$ cat Makefile
file.out: jsons
cat $(wildcard jsons/*.json) /dev/null > file.out
$ ls -R
Makefile jsons/
./jsons:
foo.json
$ make
cat jsons/foo.json /dev/null > file.out
$ make # no file mods => no-op
make: `file.out' is up to date.
$ touch jsons/bar.json
$ make # new file => re-make
cat jsons/bar.json jsons/foo.json /dev/null > file.out
$ make
make: `file.out' is up to date.
$ rm jsons/foo.json
$ make # deletion => re-make
cat jsons/bar.json /dev/null > file.out
$ rm jsons/bar.json
$ make # nothing there
cat /dev/null > file.out
$ make
make: `file.out' is up to date.
Thanks to the folks who replied! Looks like I didn't dig far enough into GNU Make. It's got a lot of the functionality I'm looking for (and I just didn't realize it). Hopefully I don't need to use it, but it even has an "eval" function for dynamically generating its own input.
Interesting! I had originally tried to do this with GNU Make (with the most naive approach you can imagine) and it wasn't able to notice that a prereq that matched the wildcard had been deleted (i.e. it thought the target was up to date, when I wanted it to rebuild).
I'll read up on the wildcard function and see if that is what I was looking for:
For example, let's say I have a rule that consumes *.json to produce some target. I want it to rebuild that target if any *.json files are added or removed (or modified).
As far as I'm aware, Make relies on file system timestamps, so even if it supported wildcards (I'm not sure if it does or not), it wouldn't be able to notice that foo.json has been deleted and a rebuild of the target is needed.
I thought I'd ask here before I go build such a tool (to support incremental rebuilds of a static site).
Edit to add that my particular use case has a non-prescriptive set of directories, nested a few levels deep. I'm actually realizing that that is probably a bigger hurdle for using something like Make (e.g. transform all *.md files, no matter how deep; but copy everything else, like images, verbatim; oh, and also then aggregate all *.md files into an Atom feed). Yes, I know this is asking a lot of something like Make!