How many deployed web apps really use Twisted? There are like 3 web packages in Twisted, most of which are really buggy, and as far as I could tell, barely used (even they acknowledge this, see http://twistedmatrix.com/trac/wiki/WebDevelopmentWithTwisted).
When we were developing this, we found that Twisted introduced as many problems as it solved in terms of incomplete features and bugs. The other protocols seem to have more attention than HTTP from what I could tell.
The other protocols seem to have more attention than HTTP from what I could tell.
I'm not sure even that is true. When I used twisted to write the justin.tv chat server (which is essentially an irc server) I gave up on twisted's irc protocol implementation and wound up just using twisted as an I/O layer.
Yah, that seems to be Twisted's problem. I think projects that don't have a real site using them day-to-day end up in this state often - lots of incomplete implementations of lots of features.
This is sort of the point. twisted isn't a web framework. It's a networking framework. I work on a lot of really awesome networking projects that either don't have web interfaces (various xmpp services) or have ones that need work (buildbot).
Instead of filling an obviously missing hole in an existing framework, a new one was created that is missing all of the stuff that the other project has.
The twisted http client stuff is quite good. I used it along with a friend to build a tool that is a realtime (in the web sense) gateway between friendfeed and xmpp clients a couple days after the realtime API was released (before friendfeed launched their own). It works very well and I still use it today.
Unfortunately, if I wanted to build a tool that did similar stuff with a Tornado front-end, I'd have to write a new HTTP client that's compatible with Tornado's event loop (unless I'm mistaken).
Everything comes at a cost. Tornado demonstrably solved Friendfeed's problems quite well, but, as I stated above, I can't use it to solve my problems because even if I rewrote my apps, I'd lose all of the rest of twisted upon which I'm relying. e.g. I can't just bolt it onto buildbot (which would be so ideal as our web interface is suffering from lack of a framwork, but the backend is really good).
It does, but from reading the code, it looks like it's closer to to twisted's getPage functionality. That's OK for things like Friendfeed's realtime API where you do long polls that return chunks of data and then process the data online and loop.
It's not OK for things like twitter's realtime APIs that provide infinite streams of data.
However, even in the case of friendfeed's API, I process the data incrementally. It's not uncommon for users of my service to receive data via xmpp before the http request has even completed.
I also use this technique for twitter's non-realtime APIs -- I use a pull-based SAX parser (that comes with twisted) to incrementally process the stream and collect and pre-sort the interesting parts. Then one of the callbacks I attach to the completion of the request (note one of: I have several that are reusable components) delivers the pre-sorted, pre-filtered results out via xmpp. I did this to reduce memory used in my daemon. It really helped.
So while it has one, it doesn't appear to be full-featured enough to work in my applications. And this is the sort of cause of confusion here. Twisted has a great network stack and a ton of really awesome protocol implementations sitting on the shelf (my apps mix http client and servers, xmpp, couchdb, dns, finger, etc...). What it doesn't have is a decent web framework.
One of the twisted guys, however, said it should be possible to transplant the web framework part of tornado onto twisted, so that would be great for the rest of us.
When we were developing this, we found that Twisted introduced as many problems as it solved in terms of incomplete features and bugs. The other protocols seem to have more attention than HTTP from what I could tell.