I love that idea! Honestly even if it doesn't show up as a sex-difference doing this same trick might be nonetheless fruitful by asking it to predict say cis woman / trans man.