RubyConf 2007 Second Day Afternoon

Ed Borasky: Profiling and Tuning Ruby 1.8

Slides are available here. Cougar is the name of the project that this is coming from?

Is Ruby 1.8 Slow?

To benchmark: Collect a set of benchmark times, then normalize them, then compute the geometric mean of the ratios.

Alioth is a popular (if controversial) set of benchmarks. Using gcc as the standard, java is about 3 times slower, python is about 10, and ruby is 19 times slower (slower than Python, PHP, and Perl).

“Hemibel thinking”: (half of a an “order of magintude” [factor of 10]). Look for hemibel improvements or difference. Any greater accuracy doesn’t really help. Hemibel ratios: log10(a/b) * 2. Now we can rethink java as ~1, perl ~2, ruby ~2.6 (in terms of hemibels). Now we can say “Ruby is sort of slow.”

But, of course, reducing the speed of a language to a single number is not really that interesting. Instead, we need to find the variation of the benchmarks. Cue Box and Whisker Plots (they show a lot of data in a small amount of space). Now yarv looks really nice, with a really limited variation over the different benchmarks (python and ruby do well, perl and php do poorly). Each language has big (upper) outliers. Ruby’s worst outlier is “spectralnorm 500” (also bad for yarv).

OK, it’s slow, now what?

What to do: Throw hardware? Wait for 1.9/JRuby/IronRuby/Rubinius? Tune for 1.8?

Let’s try tuning 1.8 against the 1.9 slightly-modified benchmark suite. First, check how much you can get of GCC optimizations. There’s something to be gained there, but it’s a pretty small difference (1/2 a hemibel). Next, check out gprof. gprof shows the top three most expensive methods were rb_eval, rb_call0, and rb_call (for slightly more than 50% of the time). Another GNU tools is gcov (for coverage and profiling), and it can provide some
more help.

Phil Hagelberg: Tightening the Feedback Loop

How can you become a more effective programer? First, read The Pragmatic Programmer. Next, test more (duh) and use Test::Unit or RSpec (don’t waste your time doing stuff by hand that the computer could automate). autotest takes away a bit of the discipline of good testing (it tells “you what you need to know when you need to know it whether or not you know you need it or not”). But, autotest still requires a lot of context switching between terminals. Consider a solution like Flymake for emacs (which does in-editor syntax checking).

Testing habits help show the general feedback principles nicely. Many other parts of software development could benefit from the same introspection, measuring, and automation. What else should we measure?

  • Accuracy (syntax highlighting and automatic indentation): modern editors
  • Meta-Accuracy (especially when starting out, are your accuracy checks actually accurate?): heckle, rcov
  • Maintainability (hard to measure, but you can measure complexity): flog
  • Performance: write your own automated test for what you care about

Now that you’re watching various things, track it over time so that you can chart personal/project progress.

Eric Hodel: Maximizing Productivity

(How to find time to contribute to more projects)

Fun is the key to high productivity. It’s easier, of course, to have more fun on stuff you want to work on rather that something that someone else wants you to work on. YAGNI & KISS help break projects into tiny, implementable pieces. Doing a lot of tests up front will help other people write patches for you and will help you not make mistakes when adding new features. [More pimping for heckle and autotest.]

Document late in the game to capture the latest changes. README should provide a quick-start, links to the rest of the documentation, a bug tracker link, and a synopsis and feature overview.

Finally, before releasing anything have a partner review your code.

One Response to “RubyConf 2007 Second Day Afternoon”

  1. Alioth is not a set of benchmarks; it’s a GForge run by the Debian project. The Computer Language Benchmarks Game is hosted at Alioth, though.