Saturday, June 20, 2009

Benchmark.pm

Tim Bunce++ !! Benchmark.pm is one of the great libraries that separates Perl from other dynamic languages. It's my favorite module for determining the performance of code snippets that run external from my full application environment.

Sometimes, you tend to need to know what your response time is at given rates of input, so you can know your boundaries on maintaining a specified level of response time. (A webserver's response time to a dynamically generated request for example). Averages, rates, etc, tend to hide the fat tails on either end of the response time spectrum (of a given rate).

Taking your result set, and looking at from a percentile perspective gives you the fat tails, an idea of what your median execution time is, and it allows you to more easily compare two different rates (say, 100 page requests per second, vs 1000 requests per second).

In my world, it's how many quotes, or orders per second, and every microsecond counts. Finding a fat tail helps us expose design flaws, technical limitations, or simply algorithms that don't scale terribly well.

So, to summarize...

Benchmark is an awesome tool for comparing snippets of code.
Calculating percentiles is a great way to understand how a system is capable of handling varying levels of load, and how "fat" the tails are to understand a systems capacity.

No comments:

Post a Comment