What happens in our heads can certainly be expressed mathematically. Since neurons are activated by electricity in a predictable way, and electrical signals can be measured and their strength expressed with numbers, any neural system including the brain can be modeled with a mathematical function that maps inputs to outputs.

But the type of conscious, methodical calculation that humans do when they are "doing math" is not involved here, and is not how GPT works.