As someone who hasn't spent a ton of time extending any CMS but uses a lot of frameworks, I enjoyed EE too during the short amount of time I used it. Out of wordpress, drupal, and EE, EE is the one where writing plugins felt like normal programming since they hooked right into code igniter. (Not sure how things have changed in the last couple of years though.)
Main downside of EE that I remember is that when you add new functionality, the way it accomplishes this is by dynamically altering a table schema to add new columns. You can easily get up to hundreds of columns. That just seemed like it had to be a wrong approach to me, but I didn't investigate further.
