I think you are mixing things. Quaternions are in the same category as complex numbers. They can be represented as matrices, and there are probably nice uses of matrices where the element is a quaternion (such as QDNNs) instead of a real number. My experience is that in massive architectures such as LLMs, simpler forms are more successful unless there is a true benefit to representing things with more elaborate types of scalars (such as in physics, or 3d graphics).