It’s the same reason LLMs don’t perform well on tabular data. (They can do fine but usually not was well as other models)

Performing feature engineering with LLMs and then storing the embeddings in a vector database also allows you to reuse the embeddings for multiple tasks (eg clustering, nearest neighbor).

Generally no one uses plain decision trees since random forest or gradient boosted trees perform better and are more robust.