DonsProxy HowTo - Using the HttpXmlSubscriber to analyze site characteristics

Follow the Basic example, but instead of using the HttpHead subscriber, use the HttpXml Subscriber:

All output will be written to HttpXmlSubscriber.xml in the current directory. When HttpXmlSubscriber stops it will finish writing the XML output. The Xml post-processing is not integrated into the GUI, but should be at some point. So, the rest of this is still command-line stuff:

The class com.moneybender.proxy.analyzers.CachingSummary is an example of how the XML can be processed using a SAX parser such as Xerces. The subscriber has already done the "hard" work of pairing together the requests and their responses. The analyzer just loops through the XML and generates a human-readable summary in html form:

./sessionsummary.sh
2008-03-09 08:23:55,505 INFO  [CachingSummary] - Processing file HttpXmlSubscriber.xml
2008-03-09 08:23:55,521 INFO  [CachingSummary] - Analyzed HttpXmlSubscriber.xml and left results in HttpXmlSubscriber.html

Since the output is in XML, there is a veritable smorgasbord of parsing tools available to process the results in whatever way you see fit. The old DonsProxy included some XSLT transformations that would convert the output to Perl and bash scripts that could be used to repeat the session for testing and monitoring of a web site. Yes, you can use XSLT, but it can get pretty squirrelly to generate Perl from XML using XSLT. To me, SAX parsing is more straightforward, but that's just a matter of preference.

For illustration, let's look at a sample Ruby on Rails application, eXPlain Project Management Tool. A quick run using under Mongrel and under Webrick will let us see any minor differences. A longer run might let us analyze performance. If you're not familiar with Ruby or Ruby on Rails, don't worry about it. The same basic process applies regardless of what you use to implement your application.

The first step is to run eXPlain under Mongrel, and collect the output:

ruby script/server

eXPlain will now be running on port 3000. Next, modify proxy.sh to listen on 3001, and forward its traffic to 3000:

Run HttpXmlSubscriber to collect the data, then use a browser to hit http://localhost:3001, and perform a few operations. Once complete, go to the HttpXml tab, and stop the proxy. The HttpXmlSubscriber will stop, and flush its data to HttpXmlSubscriber.xml, where we can analyze it. You can expect that it would look something like this: MongrelExplain.xml. Now process the data. Run CachingSummary by invoking the script ./sessionsummary.sh, and it will create an output file that looks something like this: MongrelExplain.html

We might expect to see different results if we run under Webrick, and we do, but this isn't enough data to draw any conclusions. You should run a longer test before trying to decide what it means. If you'd like to save the results from this run, copy HttpXmlSubscriber.xml and HttpXmlSubscriber.html before proceeding.

Now, stop the process running eXPlain, and restart it under Webrick:

ruby script/server webrick

Collect the data in the same manner as before. You can see my results in WebrickExplain.xml and in WebrickExplain.html

Notice that Mongrel is delivering javascript as "text/javascript" and Webrick delivers it as "application/octet-stream". Interesting. You can see that there are some performance differences, but again, there's not enough data here to draw conclusions. With a more extensive test run, though, you should be able to see items that bear investigation for improving the responsiveness of your application.