It's pretty easy to increase the grain size of a material - ceramics processing does it all the time with spray dryers. The problem comes down more to a) composition and b) cost of processing. Sand is highly naturally refined - rivers and the ocean break down almost everything but silica, and the entire erosion process acts like one giant particle size sorting machine. "clay" and "silt" etc. are purely descriptions of size; they can vary widely in chemical makeup, and most of those variations will negatively impact concretion.