There are a bunch of options to borg check depending on what you want. A full check with --verify-data is indeed rather slow, because it checks everything (essentially equivalent to seeing if all archives can be extracted plus a bunch of extra checks). If you only want to detect e.g. bit-rot, --repository-only will be sufficient in most cases and will be I/O limited.

There are certain priority to what you treat as issues.

Slowness is the least of the problem against data loss or repo corruption and perhaps you may be able to somewhat circumvent it by splitting the repo to different locations or disks.