Well, you can't have different configs for different hosts. Other than that, I can't quickly recall what other limitations are, I see none. I really like the simplicity of the "pure git" approach.
My dotfiles repo dates back to 2018, I'm happy user of this git one-liner for the past 6 years.
Might be niche but for me - I have config files outside $HOME, I use a number of `.gitignore`-aware tools (tree, fuzzy finder), and I just don't like `git status` telling me I'm in a repo in any subdir of $HOME.