Reaction to The Platform Scalability Project… Comparing Rails & PHP

Last night I attended a very interesting talk by William Sobel (founder of the fine East Bay Ruby Meetup) and Akara Sucharitakul about Olio, a project to compare the performance of web application stacks.  The initial research compared PHP5 and Rails2 and (grossly paraphrasing here) concluded that PHP supports about three times as many users as Rails (for the uncached version of the app) for the same amount of operational cost (they didn’t measure development cost).  There are dozens (maybe hundreds) of caveats with that statement, but that’s the takeaway, and it made me cringe.

The big problem I have with this experiment is that the PHP version used raw SQL, while the Rails version used ActiveRecord ORM in “out-of-the-box” mode (i.e., no tuning of the generated SQL), and that’s just not a fair comparison in a “real world” performance study.  Anyone who has tuned a real Rails application knows that out-of-the-box ActiveRecord performance sucks, especially when you move the database to another machine.  It generates a lot more SQL statements than it needs to, and these SQL statements take a lot more time than they need to.  Any professional Rails developer knows how to tune the SQL produced by ActiveRecord, which can easily increase performance by 3x or better (depending on the particulars).

A cached version of the application will be more accurate, but to really be fair they should tune the SQL generated by ActiveRecord as any experienced Rails developer would do.

Freezing rmagick in vendor/gems

You can freeze the rmagick gem in vendors/gems using the normal process (first “gem install” it on your local system, then “rake gems:unpack” to copy it into vendors/gems). RMagick requires ImageMagick to be installed first, so you’ll need to take care of that using the platform-specific way (e.g., “port install ImageMagick” on OS X).

This gem contains native (C) code which must be compiled before the app is run. I handle that in production with the following Capistrano config, but you’ll have to run “rake gems:build” manually in the development environment when this is first set up and if ImageMagick is updated (else you’ll get an error about a version mismatch):

after "deploy:symlink", "build_native"
task :build_native do
  rake = fetch(:rake, 'rake')
  rails_env = fetch(:rails_env, 'production')
  run "cd #{current_path}; #{rake} RAILS_ENV=#{rails_env} gems:build"
end

Once the gem is unpacked, you then need to point Rails at the proper library location and path to the native code with config/environment.rb:

Rails::Initializer.run do |config|
  ...
  config.gem "rmagick", :lib => "RMagick2"
  ...
  config.load_paths += %W( ... #{RAILS_ROOT}/vendor/gems/rmagick-2.8.0/ext/RMagick )
  ...
end

Finally, I tell SVN to ignore the platform-specific build artifacts (the following is what I use for OS X):

svn propset svn:ignore "bin" vendor/gems/rmagick-2.8.0
svn propset svn:ignore "extconf.h
RMagick2.bundle
mkmf.log
Makefile" vendor/gems/rmagick-2.8.0/ext/RMagick

Patching ActiveSalesforce for Rails 2.2.2

Here’s what I did to get ActiveSalesforce working under Rails 2.2.2:

1. $ gem install activerecord-activesalesforce-adapter
2. apply the following patch to lib/active_record/connection_adapters/activesalesforce_adapter.rb:

   698c698,700
   <                 referenced_klass.connection = klass.connection
   ---
   >                 def referenced_klass.connection
   >                   klass.connection
   >                 end
   705c707
   <                 klass.belongs_to referenceName.to_sym, :class_name => referenced_klass.name, :foreign_key => foreign_key, :dependent => :nullify
   ---
   >                 klass.belongs_to referenceName.to_sym, :class_name => referenced_klass.name, :foreign_key => foreign_key

3. test with the following script:

   class Contact < ActiveRecord::Base; end
   Contact.establish_connection(:adapter => "activesalesforce", :username => <YOUR USERNAME>, :password => <YOUR PASSWORD>)
   Contact.first

Enjoy.

ActionMailer::Base.register_template_extension deprecation warning in Rails 2.2.2

Rails 2.2.2 issues the following warning:

DEPRECATION WARNING: ActionMailer::Base.register_template_extension has been deprecated.Use ActionView::Base.register_template_extension instead.

Despite what Joshua Peek seems to think, ActionView::Base.register_template_extension doesn’t exist in Rails 2.2.2:

undefined method `register_template_extension' for ActionView::Base:Class (NoMethodError)

Grep for “register_template_extension” in vendor/rails to see for yourself.

I was using this method to enable HAML templates for outgoing email, but apparently this isn’t necessary as of Rails 2.2.2 (HAML 2.0.4).  So, if this warning is annoying you too, trying deleting the method call.

Nonetheless, I found the brusque response to my Lighthouse ticket pretty annoying.  Maybe Joshua hasn’t upgraded to Rails 2.2.2 yet.  (If he’s going to triage Rails bugs, he should.)

Update: The correct replacement is ActionView::Template.register_template_handler.

Rcov and vendor/gems

As a matter of practice I like to place all third-party dependencies into the Rails vendor/ directory, so rcov should go in vendor/gems.

I tried following the guidance at http://deadprogrammersociety.blogspot.com/2007/06/cruisecontrolrb-and-rcov-are-so-good.html, but ran into trouble when the rails_rcov plugin died with “no such file to load — rcov/rcovtask”.  It turns out that rails_rcov.rake requires this file but doesn’t use it.  Ok, I guess we’ll comment out that “require ‘rcov/rcovtask’” statement and carry on.

The next error is that the “rcov” executable can’t be found.  To cut to the chase, make the following substitution in the “ruby” method at rails_rcov.rake:134:

      cmd = (is_windows? ? ‘rcov.cmd’ : ’sh -c \’RUBYLIB=vendor/gems/rcov-0.8.1.2.0/lib vendor/gems/rcov-0.8.1.2.0/bin/rcov’) << ” #{RcovTestSettings.to_params} #{args}\’”

There’s got to be a cleaner way to make this work, but I need to get on with the real work at this point.

Rails 2 asset caching issue with load-balanced environments

If you use the :cache argument to javascript_include_tag and stylesheet_link_tag in a load-balanced environment, beware!

Say the first request for the Rails view gets directed to server A.  Server A will dutifully generate the cached .js file in anticipation for the forthcoming request from the browser for it.  The second request then gets directed to server B, which hasn’t yet received a request for the view or generated the cached .js file.  So, the browser gets a 404 error and your JS (or CSS) doesn’t load.

The only reliable solution I know of is to manually load one view which references each cached asset file, directly from each server in the cluster (i.e., bypass the load balancer).  We can automate this with a Capistrano “after :deploy” hook, but it still seems really clumsy.

Anyone have a better solution?

acts_as_audited and “Declare either attr_protected or attr_accessible for User, but not both.”

You’re using attr_accessible on your User model, to which you also want to audit changes using acts_as_audited.  Unfortunately it blows up with “Declare either attr_protected or attr_accessible for User, but not both.”  The problem is that acts_as_audited is adding its own attr_protected.  We can patch it in acts_as_audited.rb line 71 (as of this writing).  Change:

attr_protected :audit_ids

to

attr_protected :audit_ids if accessible_attributes.nil?

P.S. I would have commented directly on the acts_as_audited site in hopes of helping more people find this, but it appears that they’ve disabled comments.  Bummer.

Change to ActiveRecord MySQL adapter breaks tests

This patch: http://github.com/rails/rails/commit/a37546517dad9f6d9a7de6e1dba4d960909d71e8
means that test database structure is not created properly (integer columns get recreated as bigints).  If you’re also using Foreign Key Migrations, your test database simply won’t get built.  It’s too bad the Rails core team didn’t more carefully consider the implications of this patch.  Anyway, we can correct the mistake with the following patch:

===================================================================
--- vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb	(revision 62)
+++ vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb	(working copy)
@@ -470,9 +470,9 @@
         case limit
         when 0..3
           "smallint(#{limit})"
-        when 4..8
+        when 4..11
           "int(#{limit})"
-        when 9..20
+        when 12..20
           "bigint(#{limit})"
         else
           'int(11)'

READ THIS if you use the acts_as_solr Ruby on Rails plugin

I was perplexed by Solr searches for objects of a particular type that were returning objects of the wrong type until I found this. Basically, Foo.find_by_solr(…) returns objects of type Bar, frequently resulting in errors like

RuntimeError (Out of sync! The id 123 is in the Solr index but missing in the database!)

The fix is to add the line:

<field name="type_t" type="string" indexed="true" stored="true" required="true"/>

to the file vendor/plugins/acts_as_solr/solr/solr/conf/schema.xml.

This bug has been open for three months but still not addressed, despite the seriousness of it.

Amazon Associates/ECS via SOAP

Background

I want to integration with the Amazon Associates (nee ECS) web service. There are several gems available for doing this, but their functionality is limited and their documentation sucks. Amazon publishes a WSDL descriptor for their SOAP interface, so let’s see if we can roll our own.

Resources

http://crastinate.jonwiley.com/?p=15

http://www.brendonwilson.com/blog/2006/04/02/ruby-soap4r-wsdl-hell/

Preparation

1. gem install soap4r
2. wsdl2ruby.rb –wsdl http://ecs.amazonaws.com/AWSECommerceService/2007-10-29/AWSECommerceService.wsdl –type client –module_path Amazon

Code

require 'rubygems'
gem 'soap4r'
require 'defaultDriver.rb'

request = ItemLookupRequest.new
request.itemId = ["B00005JLXH"]
body = ItemLookup.new
body.aWSAccessKeyId = <YOUR AWS ID>
body.request = request

obj = AWSECommerceServicePortType.new()
response = obj.itemLookup(body)

puts response.items[0].item[0].itemAttributes.title

Rock.