Today I learned your can crawl all the pages of your website using a simple wget command:

  check-links:
   @echo "NOTE: Ensure 'make run-site-deep' is running before running this target"; echo
   -wget -r -l 0 --spider -nd -nv http://localhost:8000/ -o run.log
   grep -B1 broken run.log
   @echo Done; echo

Source: https://github.com/susam/susam.net/blob/08a2d7b/Makefile#L37...