Each arm/riscv set likely share cache and register space (which takes most of the die space by far), resulting in being unable to use them both simultaneously.
Considering that these are off-the-shelf Cortex-M designs I doubt that Raspi was able or would be allowed to do that. I'd expect most of the die to be the 512K SRAM, some of the analog and power stuff and a lot of it just bond pads.
