Why not continue to use rpm/apt for building and packaging? You can reuse these packages in the Dockerfile and the image, but not the other way around.
Because of the little gains and large additional effort required in exchange.
rpm/apt are quite involved to package and the tooling is (at least compared to OCI) limited. They would provide little advantage over OCI which works on any packaging system and even OS. They don't enjoy the ecosystem advantages of OCI, despite their age.
In summary, it could be doable, but it would require much more additional effort than packaging in OCI first.
But sure, since it can be done, maybe someone wants to take on it. I won't be down for the job myself ;)