It could have held it's own technologically, except that SEGA were complete dumbshits and their console supported burned CDRs out of the box. That was a coup de grace.
It may not have helped, but let's not call people complete dumbshits for leaving a loophole in their DRM. That was a mistake that could happen to anyone, and the outcome was decided by then anyway.
Now botching the Saturn launch because of political infighting between the Japanese and American branches of the company? If you're going to call them complete dumbshits for anything, that's what I would pick.
I had a friend with a Dreamcast he had to mod it to play CDRs as the Dreamcast had a different CD format to hold more info. They had to reduce the CD to fit in a 700M CDR. Most games didn't use the full CD.
Google mod chips some time they had them for the Playstation and XBox to use CDRs.
You didn't have to physically mod the Dreamcast console.
An exploit was found due to the console support for CD's with multimedia support, called MIL CDs. Initially a bootdisc was needed which you swapped with a CD-R, later all dumps of games were done in such a way that the exploit was bundled.
You are correct though in that some games used the full 1GB+ capacity of the GD-ROM format and assets had to be removed/compressed in some cases.