I think I'm in the same boat as you. I've discussed this with my spouse and we've determined that my mind's eye is pretty impaired. It's there, but just barely.
I totally agree with the "spatial arrangement" of math/logic etc, I'm a software engineer and arranging and abstracting things into their logical parts internally comes as easily as looking at component parts laid out on a table to me (although it's not really visual).
In regards to chess (I'm pretty decent at chess) I think it's like this, if you wrote a chess program, you wouldn't write it to visualize the representation of the board, you would set up variables to track state, etc. I think my "internal representations" of abstractions are just that, variables that my mind is tracking and understanding the relationships between, as easily as any other sense.
While looking for a way to explain my own aphantasia, I came across the works of Temple Grandin, a poster child of high-functioning autism. Since there is no physiological test for ASD, I believe that the following cognition types may be presented within the general audiences as well, however, they are not treated as outliers as long as their 'owners' have sufficient social skills. Thus, I suggest that you ignore autism/ASD references below:
"I have observed that there are three different specialized autistic/Asperger cognitive types. They are:
1) Visual thinkers such as I, who are often poor at algebra.
2) Verbal specialists who are good at talking and writing but they lack visual skills.
3) Pattern thinkers such as Daniel Tammet, who excel in math and music but may have problems with reading or writing composition." [1]
"<...>Chess masters think not in pictures or words, but are good in recognizing patterns. Pattern thinking is a more abstract form of visual thinking; thoughts are in patterns instead of photo-realistic pictures. Pattern thinkers see patterns and relationships between numbers. Some of the best descriptions are in Daniel Tammet’s book Born on a Blue Day (Tammet 2006) and in Jerry Newport’s book Mozart and the Whale (Newport et al. 2007). The weak areas in pattern thinkers is usually reading and writing composition." [1]
I totally agree with the "spatial arrangement" of math/logic etc, I'm a software engineer and arranging and abstracting things into their logical parts internally comes as easily as looking at component parts laid out on a table to me (although it's not really visual).
In regards to chess (I'm pretty decent at chess) I think it's like this, if you wrote a chess program, you wouldn't write it to visualize the representation of the board, you would set up variables to track state, etc. I think my "internal representations" of abstractions are just that, variables that my mind is tracking and understanding the relationships between, as easily as any other sense.