Sometimes I wish there were a more modern replacement for Graphviz. There are things that it doesn't handle very well (e.g., nested subgraphs) and I feel like there are good (albeit proprietary) algorithms which could make their way into a general package.
Not as easy to use, because it does not have a simple cli, but nested layouts are better than dot's with the Eclipse Layout Kernel. There is a JavaScript version, as well and you can try it in recent plantuml versions.
I believe it's an open problem what makes a curve look "natural" or how to do a good job of routing multiple splines around obstacles so they do not intersect each other, though the work of Keean Crane at CMU seems promising: https://www.cs.cmu.edu/~kmcrane/Projects/RepulsiveCurves/ind...