caboose

We’ve hit a snag deploying a JRuby application on the OC4J app server. We’re having all sorts of issues getting our application running (it works fine, but there are strange errors are deep within the stack).

I need a consultant to come onsite in San Francisco for a day or two to fix our deployment issues and help us fix or re-set up the production servers. Please contact me (courtenay @ entp.com) if you can help some time in the next 1-2 weeks.

Nginx just released a vulnerability fix. It's a nasty one, since it allows remote execution. Since RPMs don't exist yet, here's a quick-and-dirty way to roll your own on Fedora-based systems. I basically followed these guides: http://www.linuxweblog.com/patch-rebuild-rpm and http://perso.b2b2c.ca/sarrazip/dev/rpm-building-crash-course.html

Since on this server I'm running 0.6.3x, I'll be updating to that version. The fixed version is 0.6.39.

While they do the building from their own user account, a commenter suggests setting up an account specifically for building RPMs. The latter is probably preferable, but we're going to ignore that in the interests of expediency.

Assuming you never built an RPM before:

1. Install RPM build tools and setup your environment

   # yum install rpm-build

   Now, create ~/.rpmmacros with this line

   %topdir /home/youruserid/rpm

   And create a bunch of directories

   $ mkdir ~/rpm $ cd ~/rpm $ mkdir SOURCES SPECS BUILD SRPMS

2. Find the most recent nginx source package on rpmfind.net. This will give you a working recent RPM to fix.

3. Install the .src.rpm package with rpm -i nginx-0.6.38-1.src.rpm which installs the necessary files into ~/rpm/SOURCES

4. Here's the hacky part. Download nginx-0.6.39.tar.gz and copy it to ~/rpm/SOURCES. Then, edit the ~/rpm/SPECS/nginx.spec file and replace "6.38" with "6.39" wherever it occurs (just once, I believe).

   Now, build the RPM with rpmbuild -ba SPECS/nginx.spec

   On my system, I had to also install (with yum) pcre, pcre-devel, and perl-ExtUtils-Embed before nginx would build.

   Now, wait while it builds.

5. You now should have nginx-0.6.39-1.fc8.x86_64.rpm (or whatever your system is.)

   # rpm -Uvh nginx-0.6.39-1.fc8.x86_64.rpm

   # /etc/init.d/nginx restart

Done!

Munin is pretty much the standard system monitoring software for linux machines. If you have a server on EngineYard, they don't officially support munin, so it's up to you to get it installed and working. If you use a rails monitoring plugin like NewRelic, well, it's just like that, only older, open source, and uglier. We use and rely on New Relic on Tender and Lighthouse, and it's really fucking great. But for servers with no rails apps, and to get some nice aggregated graphs, I (as chief sysadmin) like using Munin.

There are two parts to munin - the server, and the client. It's sort of confusing; there's a daemon that runs on each of your machines and accepts connections from remote (munin nodes), and there's a cron job that runs on one specific machine which periodically fetches stats from each machine and generates the graphs. I'm going to assume you have munin itself set up somewhere.

First step, you need to get some ports forwarded, because your slice isn't directly accessible to the outside world. A helpful EY tech will forward your port 4949 (munin) from each slice, to something accessible on a public IP, like 22491, 22492, etc.

Now, since we're running gentoo, you'll need to install munin using the emerge package manager. On some OSes (like fedora) you install munin-node but on Gentoo it seems you get both munin and munin-node in one package.

# emerge munin

After about 15 minutes of compiling, we're good to go. You'll want to edit the /etc/munin/munin-node.conf file and add the hostname of your main munin machine as a regex. This will (hopefully) prevent bad guys from seeing your load charts.

allow ^74\.123\.111\.100$

Munin-node works by looking in a directory full of symlinks to executables. It runs the executables and records the results periodically. So, next (still as root) we want munin-node to discover what it should be running.

# munin-node-configure --shell

You'll get a dump of about 30 results that looks like

ln -s /usr/libexec/munin/plugins/cpu /etc/munin/plugins/cpu
ln -s /usr/libexec/munin/plugins/df /etc/munin/plugins/df

You can copy/paste those to create the necessary symlinks. There shouldn't be too much configuration required.

Start up munin-node

# /etc/init.d/munin-node start

And add munin-node to the scripts to be run on startup by symlinking it to the relevant place.

# cd /etc/runlevels/default
# ln -s /etc/init.d/munin-node ./

And you're done. Now, add the new nodes to the master config file and wait until the next cron runs to update them.

The very nice folks at Unspace sent me a beautiful care package today - a “handwritten” note and a huge poster. It mentioned that Future Ruby is in mid-july in Toronto, and invited me to come with my team.

I’m such a sucker for this type of marketing, so take heed ye marketers. Send me posters and tshirts!

I wish I could make it - Ruby Fringe had great press - but my visa ran out, and I can’t leave the US until it’s sorted out. I’ll be encouraging my team to go - but it’s a fair trek from the west coast

The best thing about this is the Russian glyphs they using in the title. One of the guys in the office HATES it when people abuse the backwards “R”. I will hang this poster and mock him mercilessly with its presence.

Due to the way Rails' development mode works, you need to make sure you don't define subclasses in your "concern" files like this:

# status.rb
class Status
  include "status/filtering"
end

and then

# status/filtering.rb
class Status
  module CustomAssociationExtension
  end
end

Instead, you should have a file app/models/status/custom_association_extension.rb so that rails can easily find it.

Ticket: #1339 (note: not solved)

I’d strongly advise against upgrading your application to Rails 2.3.2. I’ve hit quite a few bugs that have taken a big chunk out of several of my days, including:

• “can’t dup nilclass”, as reported here: http://groups.google.com/group/rubyonrails-core/msg/787b561d166abf53
• A strange bug in associations and rspec, http://groups.google.com/group/rspec/browse_thread/thread/7901db1b123eb93c
• Lots of upgrade issues with rspec requiring a rewrite of routing specs and erroneous missing templates (reported in their README)
• Anything that isn’t fixtures, but creates records in tests, will fail due to the Rails new way of doing nested transactions

Stick to 2.2 for a while longer – it’s not worth your time.

In our open source ticket tracker xtt, we have code that looks like this:

has_many :memberships do
  def contexts
    proxy_owner.memberships.sort.group_by &:context
  end
end

Basically, it lets people group their memberships to various projects by a context. So each user can have their own grouping for projects. If you don't have a context for a project, it shows that last, in an implicit 'etc' context. In the tests, we check for this behavior like

it "leaves nil context for last" do
  @contexts = @user.memberships.contexts
  @contexts.last.should == [nil, [memberships(:default)]]
end

However, since rails now returns aggregated association results (such as group_by or total with :group) as an ActiveSupport::OrderedHash, you don't get all those tasty array methods that Rails is so famous for (#last, #second etc).

ActiveSupport::OrderedHash is just some code that duplicates Ruby 1.9's hash functionality.

The easy solution here is to make your results into an array at some point, rewrite your tests, or send a patch to the rails team to mix in the Array monkeypatched code into OrderedHash.

it "leaves nil context for last" do
  @contexts = @user.memberships.contexts
  @contexts.to_a.last.should == [nil, [memberships(:default)]]
end

Remember all those years ago when we raised a bunch of cash for Rails documentation? Still paying out, regularly. Pratik’s docrails project is chugging along with almost reckless abandon. Boy do these guides look great!

Here are the most recent three articles, and wow, are they high quality!

Sven Fuchs wrote this definitive guide to Rails Internationalization which is for “translating your application to a single custom language other than English or for providing multi-language support in your application.”

Cássio Marques wrote the excellent ActiveRecord validations and callbacks guide, which provides an in-depth look at validations, callbacks, the whole AR lifecycle, and then tells you how to write your own. This is a ground-up guide that takes you from noob to expert.

Finally, Ryan Bigg nailed the ActiveRecord query interface to the freakin’ wall. This is the be-all and end-all of ActiveRecord guides, folks.

A job well done by all, and they’re all richer for the experience, both knowledge and hard cash. Writing guides is how I got started in rails, and I’d strongly recommend any smart cookies out there write a guide to a part of the code on which they’re not so hot. As a side note, none of these people are from the US, which is a reminder to those of us North American-centric coders that there is a huge world of smart people out there.

Hey everyone! Railsconf is just days away, and O’Reilly tell us that we have over 300 people, and about 50 of you are NOT going to the main conference – Cabooseconf only! Shit yeah!

unconference

So, here’s what’s going to happen. We’ll run this in unconference style, which means, a big board with your sessions (if you want to run them) and hackfests. If you have any experience “running” an unconference, please hit me up, I’m a few organizational hands short this year.

hackfests

If you have an open source project, our room is ripe for the using. In fact, this was my original request of O’Reilly: that we are the official (as much as this exists) place for OSS hacking. Tables will be set up in circular fashion, and I’m bringing a supply of extension cables, squids and other such power devices. However, you should bring your own too, just to be sure. Sidebar, I’d LOVE to work on altered beast at some point.

Food

Last year, we had our friends sponsor the conf, which meant free food, energy drinks, and such. Due to the fucked way that Vegas operates with unions and profit-starved hotels (hah!) we can’t provide these, and I really don’t want to funnel any more money into the corporate machine. So, unfortunately, no free food at cabooseconf.

Rock band

This was a great hit last year. We WILL be bringing the xbox and some other toys which we’ll use on the projector.

Electronic musicians

Much like our pal Zed Shaw, this year I’ve been doing a lot of music (ma46 and telesonic commander). Completely unrelated to ruby or rails. So, if you have some cool gear (monome, synths, anything) bring a piece or two and we’ll have some nightly jam sessions. I’ll probably bring my Moog, a mixer, and kaoss pad. We should be able to send out midi sync over the network and a midi backplane, and I’ll bring a nice big loud speaker to annoy the hotel with.

Today we were trying to sort out running CabooseConf during Rails Conf in Vegas. As you may recall, last year we provided free lunch and beverages for you all to drink while you hack. This year, we're doing the same thing under the auspices of O'Reilly.

My favorite energy drink of choice right now is Function Alternative Energy. In fact, I love it so much that we go through a few cases every month. It has caffeine, Yerba Mate and guarana in a great mix that doesn't give you a crazy buzz -- just laser-like focus. This is probably a topic for another day, but the important point is that we got a sponsorship from Function (free drinks!) so that we could bring a couple hundred of these and hand them out to our programmer friends and get them hooked.

Now, as it turns out, you can't just do that. Not in Vegas. You have to pay everyone in line.

It costs me $1 per drink consumed, as corkage. Then, I have to pay the Chef's union another $0.60. Per drink. As the guy at the Hilton said, at least it's better than us providing you with water. That costs $3.75 per bottle.

Now, it's not about the money. I know that there are people ready to either sponsor our little anticonf or I can throw some of my own cash at it. That's totally not the point. I have a moral objection to giving these assholes any money at all.

Oh, and a bowl of pretzels? $30. For that, I hope they let you keep the fucking bowl.

How many of you are planning to come to Railsconf and just hang out at the free caboose conf?

Hey everyone! CabooseConf 09 is on again. For those of you who don’t know, we have been running a free anti-conference at the same time as the O’Reilly Railsconf behemoth. Last year, Chad and Gina approached me after the conference and asked if we would run it again this year, but under the banner of Railsconf itself. Keep your friends close, I guess ;)

I’m a big fan of going to conferences and attending the “hallway track”. So, this year it’s officially sanctioned.. Come to Vegas, network, hack, code, and don’t pay a cent. We have conference rooms staked out at the Railsconf hotel, and we’re doing our best to organize similar perks to last year (free energy drinks, etc.)

You do have to sign up at the O’Reilly “railsconf”:http://en.oreilly.com/rails2009/ site and get a badge. Did I mention it’s free? One thing you need to do – as usual – go write some open source code! No, you can’t use something you did two months ago. No, you can’t use some code on some PHP project. This is railsconf, so go show it like you mean it, and contribute something to rails. We won’t let you in unless you do. Seriously. (*Merb also accepted)

We hope to be hosting all the hackfests.. so bring your open source ruby codes.

Over at @entp we teamed up with David Black, old school rubyist, trainer and mentor to collaborate on a training session. If you’ve been sniffing around Ruby or Rails and want to get a solid foundation, this is the session for you. If you work with people you’d like to truly understand idiomatic ruby, or just want to convert them to the one true way, this is where you should point your boss: http://entp.com/training/ The sessions will also be taught by Jeremy McAnally, who needs no introduction around here, and Rick Olson will be on hand to help take things to the next level.

They’ll be starting from the ground up, building the basics and producing a fine ruby coder out of you or your coworker (whoever happens to attend).

We’re holding it in Atlanta on the 1st of April, since that’s a pretty easy place for all you east-coasters to get to. The early bird pricing ends tomorrow, so now’s a good time to think seriously about signing up.

Oh my god, I just came up with a new phrase. Twime tracker. Shoot me now, please.

Anyway, about a year ago we started writing a “twitter meets time tracking” application, XTT, over at ENTP to track our time. It was more than that, really.. more like 37signals’ In/Out (which they hadn’t released yet), but also like a closed-group twitter-style status update, the difference being we track the times for each status.

It’s invaluable to see who’s working on what, but also, what people spend their time on.

We used XTT pretty much since day 1, and it works swimmingly for a team of 4-14 people (which is how much we’ve grown over the year). We let a limited number of beta testers use it, but it never really took off. I think this is probably because the users were typically single programmers who didn’t need to know what other people were working on.

Add to the fact that there are many other excellent time trackers out there (Harvest, for example) and I didn’t feel that I could turn XTT into a saleable product (it would require another few months of polishing, for not much of a return), so we let it languish. It works well enough for us, but other people just didn’t get it.

Finally, I decided to just open source XTT, and there it is on github: xtt. If you run a small team and want to check it out, let me know how it goes.

A few notes you might find interesting:

• Much of XTT’s timezone code was extracted and is now in Rails core itself.
• We take the user’s browser cookie timezone offset, and use that to automatically figure out their local time. We have people in timezones from the UK all the way across the US (3 different zones) and beyond, and it’s worked fine.

• It has a polymorphic notifier model called a Tendril that lets you notify Campfire or IRC (or anywhere!) when people change statuses
• It has a mostly-functioning AIM bot, with a patched version of the AIM library Net::TOC (I released this separately). By mostly-functioning i mean that. It has difficulty staying up :(
• The notifiers know that if you were on a project and then go work on something else, to notify (e.g. campfire) that you’re no longer working on that thing, just so your co-workers know.
• Each user gets their own “contexts”, which are a way of grouping projects. So, your contractors might group your projects under your company name, but the boss might group projects by client.
• Full import, export, and in-browser CSV editing of times.
• You can enter times in the past like this: [-25] for 25 minutes ago, or [-4h] for 4 hours ago, when you forgot to enter your time.

Be aware that this code is released under AGPL, which means that there are limitations on how you can use the code. The main one is you can’t run a hosted version of this unless you release the source code. Other than that, it’s a fairly liberal license.

Where to from here?

If you’re interested in helping out, go fork it and send a pull request. Here are some places that need improvement.

• There’s no setup/bootstrap
• There’s no UI on tendrils; right now it’s hard-coded and you have to add a Campfire row in console
• There are no blank-state messages (“You just signed up, here’s where to go from here”)
• Some of the time logic is just nasty (particulary chart*.html)
• There’s no limit if you accidentally forget to set “out”, on the maximum status time. So you could have a 10 hour time.. this could be limited with a user preference.
• The AIM bot goes down ALL the time. Rather, it just stops responding. I was thinking of adding something like, touching a file every time it loops; if the file gets older than 5 minutes, restart!
• While the AIM bot keeps a log of everyone’s “away” states (so you could, in theory, set someone “out” when they go offline) it’s not hooked up to anything. This could be a user preference, too.
• The AIM bot doesn’t rate-limit when it re-connects, so often it’ll just keep sending requests until it’s banned
• There’s a half-finished Jabber bot in there somewhere
• Things break when you add statuses that are in the past, but where there was a more recent status
• Some of the code is lightly tested
• I’d love to integrate with other services, such as recording times on lighthouse tickets based on user input (Attach blah to #125)
• Billable vs non-billable hours.
• Entering actual billing rates for projects/clients/contexts/people so you can actually create an invoice
• Create invoices

I look forward to how you might use this in your own company. Let me know!

If you want to file bugs, or want support, see our xTT Tender support site

There are a number of good reasons why you don't want your users providing their own CSS (for example, when theming their site). These are: taste (see: myspace) and security.

The former is pretty much your users' problem. The pages don't have to look terrible -- and in fact Myspace charges a LOT of money to do those custom movie or band pages (it's part of the service when you buy their primo ad space).

The latter, well, as it turns out there are a bunch of security vulnerabilities exposed in CSS. While these are mainly in IE, related to expressions (you can run javascript from your CSS). This means that users can steal others' sessions. So, while there are some excellent perl libraries out there for this, there hasn't been one for ruby -- until now! (at least that I could find).

So, here's my first attempt.

css_file_sanitize (github)

I stole most of the tests from LiveJournal's css sanitizing library, and rewrote the implementation in Ruby. I'd love to hear your collective feedback. It's a really lazy plugin; in fact, while it does have tests, you're best to just include the module in your model. This is a case of "it works on my machine" so send your patches!

Those of you passionate about test frameworks or writing your own testing libraries can skip this article.

The rest of you, who don’t test: you’ve heard about it, you’ve seen us ranting about it, but, if you have a web application written in Rails and you don’t write tests for it, how do you know it works? (I’m guessing: refreshing it in your browser). How do you know it works for other people, too? If the company for which you work doesn’t have a testing culture, why don’t you step up and insist? Take a weekend and learn about it. Do the peepcodes, then evangelize at your company.

I’m stunned at how many Rails development companies out there just don’t test. At all. And worse, how many people apply for Rails jobs without any tests for the code they included as part of their application. If an employer gets two resumes, one where the coder has submitted a few files of code with tests, and the other which is just rails scaffold in test/, who do you think they’ll hire?