That's what I was getting at. If you want to be sure that the human body who will be present in the office is the one doing the work, then you need to have at least one on-site interview.
If the work is fully remote, then does it really even matter? If you hire Person A to provide certain business results, but he actually contracts it out to Person B who does all the work, yet the expected business results are provided, then do you actually care?
1- You don't want an incompetent, non-trustworthy person to work for you. They ruin the culture and affect the entire company.
2- Once they are in, it's not likely that they care as much as the interview time to delegate the job to a person.
3- You might have other non-technical requirements, like a background check, culture fit, personal skills, etc. The person they might delegate the job has not gone through this filter.