Monday, July 31, 2006

BNL 05 - Multiple Contexts

01 - Introduction
02 - The Problem
03 - The Solution
04 - DAMP BNLs
05 - Multiple Contexts

Multiple Contexts

In 'The Solution' we proved our concept by replacing the existing system with one that allows the business logic to be written by the subject matter experts. However, the solution we provided has a few limitations our client would like us to overcome:
It can only be run from the command line, they would like a web interface
It only executes bonus calculations for the entire group; they would like to execute individual bonuses
It does not scale well; they would like the burden of calculation to be transferred to their database server.

The first thing we are going to do to accommodate these requirements is create a table to store the bonus logic.
class AddBonusLogic < ActiveRecord::Migration
def self.up
create_table :logic do |table|
table.column :script, :text
table.column :employee, :string
end
end

def self.down
drop_table :logic
end
end
We are also going to need a few views, a model, and a controller to allow the business users to add new bonus logic and to edit existing bonus logic.

File: app/views/logic/new.rhtml
<%= start_form_tag %>
Bonus Logic

<%= text_area 'logic', 'script', 'style'=>'width:90%' %>


For

<%= text_field 'logic', 'employee' %>
<%= submit_tag 'Save' %>
<%= end_form_tag %>
File: app/views/logic/edit.rhtml

<%= start_form_tag %>
Bonus Logic

<%= text_area 'logic', 'script', 'style'=>'width:90%' %>


For

<%= text_field 'logic', 'employee' %>
<%= submit_tag 'Save' %>
<%= end_form_tag %>


File: app/views/logic/list.rhtml
Bonuses: - <%= link_to 'new', :action=>:new %> <%= link_to 'execute all', :action=>:execute_all %>

<% @all_logic.each do |logic| %>
<%= logic.employee %>
<%= link_to 'edit', :action=>:edit, :id=>logic.id %>
<%= link_to 'execute', :action=>:execute, :id=>logic.id %>


<%= logic.script %>

<% end -%>


File: app/controllers/logic_controller.rb
class LogicController < ActionController::Base
def new
if request.get?
@logic = Logic.new
else
Logic.create(params[:logic])
redirect_to :action=>:list
end
end

def list
@all_logic = Logic.find :all
end

def edit
if request.get?
@logic = Logic.find params[:id]
else
Logic.find(params[:id]).update_attributes(params[:logic])
redirect_to :action=>:list
end
end

def execute
@bonus = BonusCalculationContext.evaluate(Logic.find(params[:id]))
end

def execute_all
@bonuses = []
Logic.find(:all).each do |logic|
@bonuses << SqlCalculationContext.evaluate(logic)
end
@bonuses
end
end
File: app/models/logic.rb
class Logic < ActiveRecord::Base
end


Next we need to save all the bonus logic to the database. We can copy and paste the logic from the bonus_logic folder into the web interface.

When you are done pasting the list page should show the following logic:

Jackie Johnson
apply bonus of the total profit times one percent if the current month is equal to february
apply bonus of the drug profit times two percent if the current month is equal to february


Joe Noone
apply bonus of ten thousand dollars if the total profit is less than one million dollars and the current month is equal to march
apply bonus of ten thousand dollars if the total profit is equal to one million dollars and the current month is equal to march
apply bonus of the total profit times one percent if the total profit is greater than to one million dollars and the current month is equal to march


John Jones
apply bonus of ten thousand dollars if the total profit is greater than one million dollars and the current month is equal to january
apply bonus of ten thousand dollars if the total profit is greater than two million dollars and the current month is equal to january
apply bonus of ten thousand dollars if the total profit is greater than three million dollars and the current month is equal to january
apply bonus of the toothbrush profit times five percent if the current month is equal to january

The list page also has an execute link that links to a page that doesn't yet exist. The execute.rhtml file is a simple page that shows the employee's name and their calculated bonus:

File: app/views/logic/execute.rhtml
<%= "#{@bonus.who} bonus: #{number_to_currency(@bonus.amount.to_f*0.01)}" %>
A brief look in the logic_controller reveals that the majority of the work in the execute method is being done in the BonusCalculationContext class. The BonusCalculationContext class has slightly changed to allow you to pass a Logic instance, instead of a path, to evaluate. Other than the change in how the employee name and bonus logic are stored the class is basically the same:

File: app/models/bonus_calculation_context.rb
class BonusCalculationContext
include Reloadable
extend Verbosity
bubbles :than, :is, :profit, :the, :to, :of, :bonus
numerics :thousand=>1000, :million=>1000000,
:>1, :two=>2, :three=>3, :five=>5, :ten=>10
operations :greater=>">", :equal=>"==", :times=>"*", :less=>"<"
constants :dollars=>100, :percent=>0.01, :january=>7, :february=>7, :march=>7
attr_accessor :employee_name

def initialize
@bonus_amount = 0
end

def self.evaluate(logic)
context = self.new
context.employee_name = logic.employee
logic.script.split(/\n/).each { |spec| context.instance_eval(spec) }
context.resulting_bonus
end

def last_profit
@last_year_profit ||= Profit.find :first
end

def total(arg)
result = eval "#{last_profit.total} #{arg}"
return result.round if result.respond_to? :round
result
end

def month(arg)
".month #{arg}"
end

def current(arg)
eval "Time.now#{arg}"
end

def toothbrush(arg)
eval("#{last_profit.toothbrush_in_cents} #{arg}").round
end

def drug(arg)
eval("#{last_profit.drug_in_cents} #{arg}").round
end

def apply(amount)
@bonus_amount += amount
end

def resulting_bonus
Bonus.new(@employee_name, @bonus_amount)
end
end
At this point we have fulfilled the first two of the new requirements. A quick check shows that the logic is still producing the correct results for each employee's bonus (and now the results are formatted nicely thanks to the rails helper method number_to_currency).

Jackie Johnson bonus: $135,517.02
John Jones bonus: $92,525.00
Joe Noone bonus: $53,509.01

To accomidate the last requirement we are going to execute our bonus logic in another context. One advantage to expressing your business rules in a Domain Specific Language is the ability to execute them in various contexts. By executing the DSL in various contexts you can generate multiple behaviors from the same business logic. When the rule changes over time, all parts of the system that reference the rule will also be changed.

The execute_all.rhtml file displays the results of executing the bonus logic for all employees.

File: app/views/logic/execute_all.rhtml
<% @bonuses.each do |bonus| -%>
<%= "#{bonus.who} bonus: #{number_to_currency(bonus.amount.to_f*0.01)}" %>

<% end -%>
The execute_all.rhtml view uses the execute_all method of the (previously shown) LogicController. The execute_all method uses SqlCalculationContext.evaluate to delegate the calculations to the database server. For the example I'm using Postgres 8.1.4.

File: app/models/sql_calculation_context.rb
class SqlCalculationContext
extend Verbosity
append :sql_string,
:apply=>'select',
:total=>'(select toothbrush_in_cents + drug_in_cents from profits)',
:drug=>'(select drug_in_cents from profits)',
:toothbrush=>'(select toothbrush_in_cents from profits)',
:>1, :two=>2, :three=>3, :five=>5, :ten=>10,
:percent=>'* .01', :thousand=> '* 100000', :million=>'* 100000000',
:if=>'where', :and=>'and',
:month=>1,
:january=>1, :february=>1, :march=>1,
:times=>'*', :equal=>'=', :greater=>'>', :less=>'<'
return_self :bonus, :of, :the, :profit, :current, :is, :to, :dollars, :than

attr_accessor :employee_name, :bonus_amount

def self.evaluate(logic)
context = self.new
context.employee_name = logic.employee
sql_statements = []
logic.script.split(/\n/).each do |spec|
sql_statements << context.instance_eval(spec.gsub(' ','.')).sql_string
context.clear_sql_string!
end
sql_statements = sql_statements.collect { |sql| "coalesce((#{sql}),0)"}
complete_sql = "select " + sql_statements.join("+")
context.bonus_amount = execute(complete_sql).result.flatten[0].to_f.round
context.resulting_bonus
end

def execute(sql)
ActiveRecord::Base.connection.execute(sql)
end

def resulting_bonus
Bonus.new(employee_name, bonus_amount)
end
end
The SqlCalculationContext makes use of two new methods, append and return_self, that were added to the Verbosity module.

File: app/models/verbosity.rb
module Verbosity

def bubbles(*methods)
methods.each do |method|
define_method(method) { |args| args }
end
end

def numerics(hash)
hash.each_pair do |name, multiplier|
define_method(name) { |args| multiplier * args }
end
end

def operations(hash)
hash.each_pair do |name, operator|
define_method(name) { |args| "#{operator} #{args}" }
end
end

def constants(hash)
hash.each_pair do |name, value|
define_method(name) { |args| value }
end
end

def append(var, hash)
eval "define_method(:#{var}) { @#{var} ||= String.new }"
eval "define_method(:clear_#{var}!) { @#{var} = String.new }"
hash.each_pair do |name, value|
eval "define_method(:#{name}) { @#{var} = #{var} + '#{value} '; self }"
end
end

def return_self(*methods)
methods.each do |method|
define_method(method) { self }
end
end

end
After these additions you can use the 'execute all' link on the list page to view the results. As expected, the correct results are displayed:

Jackie Johnson bonus: $135,517.02
John Jones bonus: $92,525.00
Joe Noone bonus: $53,509.01

This doesn't appear very impressive since we already knew how to calculate the results. However, the interesting thing is that instead of calculating the results using ruby, the results are calculated from generated sql statements. The sql that executes the results for each employee is generated by the SqlCalculationContext.
select coalesce((select (select toothbrush_in_cents + drug_in_cents from profits) * 1 * .01 where 1 = 1 ),0)+coalesce((select (select drug_in_cents from profits) * 2 * .01 where 1 = 1 ),0)

BNL 04 - DAMP BNLs

01 - Introduction
02 - The Problem
03 - The Solution
04 - DAMP BNLs
05 - Multiple Contexts

DRY code, DAMP Business Natural Languages

Don't Repeat Yourself (DRY) is a best practice in software development. Specifying behavior once and only once is the essence of the DRY rule. When you follow this rule the maintainability of the software is increases. The rule helps ensure that if you are required to make a change you can be sure that you will only need to change the software in one location. The benefits of this rule should be quite obvious; however, does it also apply to Business Natural Languages?

Based on my experience I believe that the DRY rule does not apply to Business Natural Languages. A major reason for using a Business Natural Language is to separate the business logic from the complexities of the under-lying system. When using a Business Natural Language, business users who are the most familiar with the domain can maintain the business logic. To a business user, a Business Natural Language should be no different than a group of phrases that describe the rules for running the business correctly.

A well-designed Business Natural Language will appear as Descriptive And Meaningful Phrases (DAMP).

In 'The Solution' I used the following business logic to calculate Jackie Johnson's bonus:

apply bonus of the total profit times one percent if the current month is equal to february
apply bonus of the drug profit times two percent if the current month is equal to february


In the Verbosity module I defined a bubble method that created methods for each member of the list passed to bubble. Each method that was defined by bubble took one value and returned the same exact value. This allowed the Business Natural Language to be verbose enough that not every word was required to have meaning. In fact 'than', 'is', 'profit', 'the', 'to', 'of', and 'bonus' are all bubble methods. Without bubble methods the business logic would read:

apply total times one percent if current month equal february
apply drug times two percent if current month equal february

The new logic can be considered more DRY; however, much of the readability has been lost at the expense of removing the some of the duplication. You should not trade readability for conciseness when designing Business Natural Languages.

As businesses change their software requires change. If software can be understood and altered by subject matter experts the business can be more responsive to change. Business Natural Languages facilitate empowering subject matter experts by using a meaningful syntax to express business logic. This meaningful syntax often contains words that do not carry programmatic value; however, they increase the overall readability of the phrase. Therefore, the 'bubble' words carry maintainability value for the subject matter experts. If the 'bubble' words are removed the logic becomes less maintainable.

It is also important to note that employees often change roles within an organization. Because staffing does change, you should not design a Business Natural Language that sacrifices readability for conciseness simply because the current subject matter expert is able to comprehend the concise version. Instead, you should design the Business Natural Language to be easily understood by any subject matter expert placed in that role.

Some programmers object to 'bubble' words because they do not effect program execution. This is a fairly common thought because programmers are still thinking in terms of 'what will make the system work' instead of trying to solve a more valuable question: How can I empower a subject matter expert to make the system work? When you begin to think in these terms you see that 'the' may be equivalent to white space to the computer; however, it's highly important to the subject matter expert who will need to continually understand and update the system.

Another concern is that 'bubble' words can confuse the subject matter expert because they have no meaning. The problem with this concern is that it assumes that the subject matter expert understands that the other words do have meaning. The 'bubble' words make much more sense when you understand that to a subject matter expert the entire phrase is important, not the individual words.

You could argue that you should allow any non-keyword to be interpreted as a 'bubble' word. There are technical issues associated with this; however, I think a more important issue to consider is that this will allow subject matter experts to write rules without following a pre-defined convention. The lack of convention could lead to decreased maintainability.

The main goal of a DAMP Business Natural Language is empowering subject matter experts by providing the most maintainable syntax available. A Business Natural Language clearly has room for improvement in the syntax if it requires training to understand. An ideal Business Natural Language contains phrases that are descriptive and meaningful enough that they require no explanation at all.

BNL 03 - The Solution

01 - Introduction
02 - The Problem
03 - The Solution
04 - DAMP BNLs
05 - Multiple Contexts

The Solution
A Business Natural Language should be used to specify expected behavior. The language should be comprised of descriptive and maintainable phrases. The structure of the language should be simple, yet verbose. Imagine each line of the specification as one complete sentence. Because a Business Natural Language resembles a complete language it is important to limit the scope of the problem being solved. An application's requirements can often be split to categories that specify similar functionality. Each category can be a candidate for a simple Business Specific Language. Lucky for us, our current application has a very limited scope: calculating bonuses. Therefore, our application will only require one Business Natural Language.

We've already seen how the requirements were originally specified:

John Jones
bonuses: $10,000 if the company posts a profit of 1 million
$20,000 if the company posts a profit of 2 million
$30,000 if the company posts a profit of 3 million or more
5% of toothbrush profits
bonus pay month: January


And, how these requirements can be specified in ruby:
if Time.now.month == 1
profit = Profit.find(:first)
bonus = profit.total > 100000000 ? 1000000 : 0
bonus += 1000000 if profit.total > 200000000
bonus += 1000000 if profit.total > 300000000
bonus += profit.toothbrush_in_cents * 0.05

File.open(bonus_log,'a') do |file|
file << "Jim Jones bonus: #{bonus.round} cents\n"
end
end
If we developed a Business Natural Language the requirements could be specified as:

apply bonus of ten thousand dollars if the total profit is greater than one million dollars and the current month is equal to january
apply bonus of ten thousand dollars if the total profit is greater than two million dollars and the current month is equal to january
apply bonus of ten thousand dollars if the total profit is greater than three million dollars and the current month is equal to january
apply bonus of the toothbrush profit times five percent if the current month is equal to january

There isn't any advantage to the above requirements as compared to the original requirements when judged as requirements alone; however, there would be a high amount of value if the requirements could also be executed as part of the application. This is exactly the power that a Business Natural Language provides you.

Lets change our existing application to work with our Business Natural Language. The first thing we can do is create a folder under lib called bonus_logic. Our new bonus_logic folder will be the folder where the subject matter experts will drop their files that determine how bonuses are calculated.

Because we previously executed the legacy application we know that John Jones bonus for last year was: $92,525.00. Lets drop a file called john_jones.txt into our new bonus_logic folder and see what we need to do to get our application to execute and produce the same result. Remember, the john_jones.txt file contains our specifications:

File: lib/bonus_logic/john_jones.txt
apply bonus of ten thousand dollars if the total profit is greater than one million dollars and the current month is equal to january
apply bonus of ten thousand dollars if the total profit is greater than two million dollars and the current month is equal to january
apply bonus of ten thousand dollars if the total profit is greater than three million dollars and the current month is equal to january
apply bonus of the toothbrush profit times five percent if the current month is equal to january

The rake task will need to be updated to evaluate all files found in bonus_logic:

File: lib/tasks/payroll.rake
namespace :payroll do
desc "run payroll"
task :run => :environment do
output_folder = "#{File.expand_path(File.dirname(__FILE__))}/../../tmp/"
output_file = "#{Time.now.year}-#{Time.now.month}-bonus.log"
output_path = output_folder + output_file
bonus_logic = "#{File.expand_path(File.dirname(__FILE__))}/../bonus_logic/"
File.open(output_file,'w') {}
Dir[bonus_logic+"*.txt"].each do |bonus_spec|
bonus = BonusCalculationContext.evaluate(bonus_spec)
if bonus.applies?
File.open(output_file,'a') { |file| file << "#{bonus.data}\n" }
end
end
end
end
The heart of any Business Natural Language are the context classes that understand execution behavior. However, before we jump right into the context class we should understand the verbosity module. The verbosity module is a module that lets you easily define some keywords of your Business Natural Language. For example, if you wanted to use 'january' in your language you could define the january method. Or, you could simply use the constants class method of the verbosity module. Verbosity has 4 methods:

bubbles, Bubble methods are methods that take an argument and simply return this argument. They are equivalent to:
def the(arg)
arg
end
Bubble methods allow you to add verbosity further making your language read like a natural language.
usage: bubbles :the, :is, :to

constants, Constant methods are methods that take no arguments and return a constant value. They are equivalent to:
def january
1
end
Constant methods allow you to easily define methods that will always return the same value.

operations, Operation methods are methods that take an argument and append the operator of an operation. They are equivalent to:
def equal(arg)
"== #{arg}"
end
Operation methods are generally used to specify the operator and right value for an operation. A method to the left an operation method will generally specify the left value and execute an eval.

numerics, Numeric methods are methods that are the word representation of a number. They are equivalent to:
def two(arg)
2 * arg
end
Numeric methods are generally used when chaining words together such as 'ten thousand'.

The implementation of Verbosity is quite simple.

File: app/models/verbosity.rb
module Verbosity

def bubbles(*methods)
methods.each do |method|
define_method(method) { |args| args }
end
end

def numerics(hash)
hash.each_pair do |name, multiplier|
define_method(name) { |args| multiplier * args }
end
end

def operations(hash)
hash.each_pair do |name, operator|
define_method(name) { |args| "#{operator} #{args}" }
end
end

def constants(hash)
hash.each_pair do |name, value|
define_method(name) { |args| value }
end
end

end
The real value of Verbosity is hard to see without seeing its usage in BonusCalculationContext. BonusCalculationContext contains the behavior that transforms the Business Natural Langauge to executable code.

File: app/models/bonus_calculation_context.rb
class BonusCalculationContext
extend Verbosity
bubbles :than, :is, :profit, :the, :to, :of, :bonus
numerics :thousand=>1000, :million=>1000000,
:>1, :two=>2, :three=>3, :five=>5, :ten=>10
operations :greater=>">", :equal=>"==", :times=>"*", :less=>"<"
constants :dollars=>100, :percent=>0.01, :january=>7, :february=>7, :march=>7

def initialize
@bonus_amount = 0
end

def self.evaluate(path_to_bnl)
context = BonusCalculationContext.new
context.extract_name(path_to_bnl)
specifications = File.readlines(path_to_bnl)
specifications.each { |spec| context.instance_eval(spec) }
context.resulting_bonus
end

def last_profit
@last_year_profit ||= Profit.find :first
end

def total(arg)
result = eval "#{last_profit.total} #{arg}"
return result.round if result.respond_to? :round
result
end

def month(arg)
".month #{arg}"
end

def current(arg)
eval "Time.now#{arg}"
end

def toothbrush(arg)
eval("#{last_profit.toothbrush_in_cents} #{arg}").round
end

def drug(arg)
eval("#{last_profit.drug_in_cents} #{arg}").round
end

def apply(amount)
@bonus_amount += amount
end

def extract_name(path)
name_parts = File.basename(path).chomp!('.txt').split('_')
name_parts = name_parts.collect { |part| part.capitalize }
@employee_name = name_parts.join(" ")
end

def resulting_bonus
Bonus.new(@employee_name, @bonus_amount)
end
end
At only 60 lines, BonusCalculationContext isn't extremely complex. The common entry point to BonusCalculationContext is a call to the evaluate class method. The class method evaluate creates a new instance of BonusCalculationContext, extracts the employee name, and loops through each line of the argument file. As each line from the argument file is executed the methods of BonusCalculationContext are being executed and making changes to the bonus_amount instance variable. When BonusCalculationContext.evaluate is finished processing each line it will return the resulting_bonus generated by the BonusCalculationContext instance. The resulting_bonus is a simple class used to hold bonus information.

File: app/models/bonus.rb
class Bonus
def initialize(who, amount)
@who = who
@amount = amount
end

def applies?
amount > 0
end

def data
"#{who} bonus: #{amount} cents"
end

private
attr_reader :who, :amount
end
At this point we return to the command line and execute 'rake payroll:run'. You should be delighted to see the tmp/2006-7-bonus.log does have John Jones bonus information. [Note: If you aren't seeing the value, ensure that the :january constant is set to the current month; otherwise the if statment will exclude John Jones' information]

Next, Lets convert Jackie Johnson and Joe Noone's business logic to our Business Natural Langauge.

File: lib/bonus_logic/jackie_johnson.txt
apply bonus of the total profit times one percent if the current month is equal to february
apply bonus of the drug profit times two percent if the current month is equal to february


File: lib/bonus_logic/joe_noone.txt
apply bonus of ten thousand dollars if the total profit is less than one million dollars and the current month is equal to march
apply bonus of ten thousand dollars if the total profit is equal to one million dollars and the current month is equal to march
apply bonus of the total profit times one percent if the total profit is greater than to one million dollars and the current month is equal to march


After dropping Jackie and Joe's specifications into the bonus_logic folder you can run 'rake payroll:run' again to see their results. A quick verification check shows that the results are accurate. We still have a lot of work to do, but we've proven our concept. Our application behavior is entirely dependent on the bonus specifications that can be altered by our subject matter experts.

In the upcoming chapters we will discuss moving our application to the web, putting the specifications in a database instead of using flat files, providing syntax checking and interactive recommendations, and many other concepts that will take us through developing a realistic Business Natural Language application.

BNL 02 - The Problem

01 - Introduction
02 - The Problem
03 - The Solution
04 - DAMP BNLs
05 - Multiple Contexts

The Problem
To show the value of Business Natural Languages we will start with an existing ruby application and then introduce the Business Natural Language. This approach should provide a good example of how to include a Business Natural Language in an application.

For the sample application imagine you've been contracted to replace a bonus calculation system for a pharmacy. The pharmacy chose to custom develop the software for calculating bonuses because of the varying bonus packages. They have given you profiles of 3 employees and the business rules required to generate bonus compensation.

Profiles:
John Jones
bonuses: $10,000 if the company posts a profit of 1 million
$20,000 if the company posts a profit of 2 million
$30,000 if the company posts a profit of 3 million or more
5% of toothbrush profits
bonus pay month: January


Jackie Johnson
bonuses: 1% of profits
2% of drug profits
bonus pay month: February

Joe Noone
bonuses: 1% of profits or $10,000, whichever is greater
bonus pay month: March

The application need only be run monthly; therefore it is okay if the process is executed manually. The existing application is executed as a rake task. It also has every employee's information hard-coded as rules in the application. Last years profit information is available in the data warehouse. For our purposes lets add some dummy warehouse data. This migration should give you the all the data you will need:
class Profit < ActiveRecord::Base
end

class WarehouseData < ActiveRecord::Migration
def self.up
create_table :profits do |table|
table.column :toothbrush_in_cents, :integer
table.column :drug_in_cents, :integer
end

Profit.new(:toothbrush_in_cents=>125050000, :drug_in_cents=>410040050).save

end

def self.down
drop_table :profits
end
end
The rake task that executes the current payroll process is pretty straightforward:

File: lib/tasks/payroll.rake
namespace :payroll do
desc "run payroll"
task :run => :environment do
output_file = "#{File.expand_path(File.dirname(__FILE__))}/../../tmp/#{Time.now.year}-#{Time.now.month}-bonus.log"
File.open(output_file,'w') {}
[JohnJones, JackieJohnson, JoeNoone].each do |employee|
employee.append_bonus_to(output_file)
end
end
end
Each employee ruby file is also fairly straightforward. They contain the logic to calculate a bonus, expressed as ruby.

File: app/models/john_jones.rb
class JohnJones
def self.append_bonus_to(bonus_log)
if Time.now.month == 7
profit = Profit.find(:first)
bonus = profit.total > 100000000 ? 1000000 : 0
bonus += 1000000 if profit.total > 200000000
bonus += 1000000 if profit.total > 300000000
bonus += profit.toothbrush_in_cents * 0.05

File.open(bonus_log,'a') do |file|
file << "Jim Jones bonus: #{bonus.round} cents\n"
end
end
end
end
File: app/models/jackie_johnson.rb
class JackieJohnson
def self.append_bonus_to(bonus_log)
if Time.now.month == 7
profit = Profit.find(:first)
bonus = profit.total * 0.01
bonus += profit.drug_in_cents * 0.02

File.open(bonus_log,'a') do |file|
file << "Jackie Johnson bonus: #{bonus.round} cents\n"
end
end
end
end
File: app/models/joe_noone.rb
class JoeNoone
def self.append_bonus_to(bonus_log)
if Time.now.month == 7
profit = Profit.find(:first)
bonus = (profit.total * 0.01).round

File.open(bonus_log,'a') do |file|
file << "Joe Noone bonus: #{bonus > 1000000 ? bonus : 1000000} cents\n"
end
end
end
end
Profit is a simple ActiveRecord::Base inheritor with total being it's only behavior

File: app/models/profit.rb
class Profit < ActiveRecord::Base
def total
toothbrush_in_cents + drug_in_cents
end
end
The existing application works; however, the level of maintainability is low. Unfortunately, employees are required to re-negotiate their bonus structure every year. Following these negotiations a programmer must be involved to change the employees bonus logic. Also, any time a new employee is hired, a new file must be introduced into the system to calculate that employee's bonus.

BNL 01 - Introduction

01 - Introduction
02 - The Problem
03 - The Solution
04 - DAMP BNLs
05 - Multiple Contexts

Introduction
Why use a Business Natural Language?

Since the introduction of computers to the general workforce businesses have searched for a solution that will enable subject matter experts to specify the business logic of an application. This solution is highly sought after since it will allow the application to be changed without the assistance of a programmer. Programmers are still required to create the application; however, the application is written in a way that empowers the subject matter experts to maintain the business logic. Enabling the subject matter expert greatly increases efficiency of maintaining an application as the needs of the business change.

Using a Domain Specific Language (DSL) is the most recent solution to this problem. A Domain Specific Language can be defined as a language created to describe a specific set of tasks. A Domain Specific Language is often used to define a specification or business rule without using a general purpose programming language such as Ruby or C#. Domain Specific Languages have been around under various names for many years. In fact, it's rare in today's business world to find a subject matter expert who isn't fluent in the Domain Specific Language in which they define their spreadsheet macros.

Unfortunately, there is a problem with the term Domain Specific Language. Programmers commonly know YACC as a DSL. Spreadsheet macros are widely used within the business community. Recently Rake and Ruby on Rails have gained popularity as DSLs. Some even consider Cobol to be a DSL. Thus the problem, you can make a case for almost anything to be considered a DSL.

Is that a really a problem? Yes, it turns out it is a problem. When we are unable to be specific what type of Domain Specific Language we are using we are forced to eliminate types of DSLs that are not relevant to the current conversation. This elimination process often slows discussions down. Or worse, people dismiss the conversations because they don't believe that DSLs can work in the way described. This, unfortunately, is fairly common because of DSLs of the past that have failed.

Because of the previous reason I think it's clear that there is a need classify the different types of Domain Specific Languages. I believe using specific classifications will allow conversations to become more productive. The primary focus of this book is enabling subject matter experts to specify the business logic in a natural language. For several months I used the term Business Domain Specific Language to describe this type of work; unfortunately, the business prefix was generally lost in conversation and the same issues emerged. Because the prefix was not effective at solving the problem I chose to coin the term Business Natural Language (BNL).

A Business Natural Language is a Domain Specific Language; however, not all Domain Specific Languages are Business Natural Languages. Business Natural Languages use natural language to represent business logic. Business Natural Languages are expressed as descriptive and maintainable phrases.
For example, a marketing executive for an airline could specify point award descriptions as:
award 1 point for each mile flown where the total flight length is greater than 500 miles
award 500 points for each flight where the total flight length is less than or equal to 500 miles

A Business Natural Language is a specific type of Domain Specific Language that will not contain unnecessary commas, semi-colons or any other constraint imposed by a general-purpose programming language. Additionally, a general-purpose language should not influence the structure of a Business Natural Language phrase. Any domain expert, with no explanation required, can read a well-written Business Natural Language as if it were simply a phrase specifying logic.

The previous airline example appears to be a specification written by a business analyst. The specification could be used to describe business logic to be implemented in a general-purpose language. However, when using a Business Natural Language the above example is a specification, but also much more. The above example is executable code, which will be used to determine point allocation after each flight flown. The above example is also documentation of the business rules contained in the point allocation application. And, the above example can be used to formulate a test case to verify the system works as expected.

Business Natural Languages allow you to specify, in one location, exactly how your application should work. By reducing duplication the maintainability of the system is greatly increased. When using a Business Natural Language the lines are not blurred between the specification and code, there are no lines. The specification is the code.

When should you use a Business Natural Language?

Business Natural Languages are ideal for systems where the behavior often changes. Generally the value of these types of systems can be directly linked to their ability to be quickly changed. In most traditional applications the business rules are specified in a general-purpose language. Changes to this style of application generally require a programmer to understand and alter the existing code base. However, a superior approach allows the behavior of the system to be specified in a Business Natural Language. A Business Natural Language is an obvious choice for this scenario because it removes the programmer from the equation. Instead, it allows the behavior of the system to be altered by a subject matter expert.

Frequently changing systems are not the only applications that can benefit from Business Natural Languages. Systems that contain a large number of similar business logic are also ideal candidates. For example, imagine a payroll system that is used to compensate all of your sales employees. In this payroll system each employee has specific variables that determine their compensation for each pay period. Their pay could be altered by variables such as the amount of business sold, profit margin per transaction, years with the company, etc. Each employee could have their own negotiated parameters for determining their pay:

employee John Doe
compensate .10 times this months sales
compensate 15 times the number yeas employed
compensate 1.10 percent if this months sales profit margin is greater than 30 percent
compensate 1.15 percent if this months sales profit margin is greater than 40 percent
compensate 1.20 percent if this months sales profit margin is greater than 50 percent

When a Business Natural Language is used in this type of system it's very easy to add a new employee into the system and ensure that their compensation will directly match exactly what their offer letter details.

When else is a Business Natural Language appropriate? Business Natural Languages are ideal for any application that contains logic specific to the business. The more business specific logic you can abstract out of an application the less programmer involvement you will need to alter the business logic. Ideally all business specific logic can be contained in one or several Business Natural Languages within your application. By enabling the subject matter experts to maintain the business rules of the application the teams can work together to quickly develop a robust application.

I was recently involved in developing an application that was estimated to take 6-8 months using a traditional development approach. Because of time to market issues this was not a viable plan. The alternative approach, using a Business Natural Language, allowed the general application to be written by the developers while the subject matter experts wrote the business logic. The application, including business logic, was complete in 4 months. During the 4 months the programmers and subject matter experts worked in parallel constantly evolving the application. During the development the subject matter experts found they needed more flexibility in their Business Natural Language. Because the teams were working in parallel the programmers were able to immediately expand the boundaries of the Business Natural Language to meet the needs of the subject matter experts.

These types of stories can vary in value due to the fact that we will never know how long it would have taken to actually complete the task in a traditional manner. However, what is measurable is the level of effort that the application requires to change the business logic. For this application the logic of the business can change on a monthly basis. If the rules were written in a language that required a programmer to make the change, each month a change request and prioritization session would need to occur. The customer for this application stated that this process generally took 2 to 3 days. Following the change, regression testing would need to occur to verify that the programmer correctly implemented the change. Compared to what we actually delivered, this process is greatly inefficient. Because we chose to use a Business Natural Language, a change could be made to the business logic directly by the subject matter expert. We also provided a test environment to run the proposed business logic. This allowed the subject matter expert to make a change, execute, and verify without any programmer support. The result, in his words was "A change that used to take 3-4 days now takes about 10 minutes."

Traditional applications waste precious time in various ways. While developing software, much time is spent educating programmers on every minute detail of the business. With this knowledge, programmers are busy developing code in a general-purpose language that mirrors the specifications given by subject matter experts. Following completion of the programming task there are often issues discovered. This cycle repeats until the program executes producing correct results. This common development loop is a time consuming process. Systems that are able to introduce Business Natural Languages avoid this loop entirely by empowering the subject matter experts to create, test, and approve the business logic with little to no assistance.

Clearly, the decision to use a Business Natural Language depends greatly on the ability of the business to provide at least one subject matter expert. At first glance it appears that a Business Natural Language creates more work for subject matter experts; however, it is important to remember that traditional application development requires subject matter experts to write specifications for the application being developed. Business Natural Languages also require the subject matter experts to create specifications, but by following some simple conventions the specifications are used within the application as actual business logic. Furthermore, anyone who has been involved in conversations where subject matter experts explain business logic to programmers has seen how challenging it can be to bring everyone's understanding to the same level. Also, even following the conversation the programmer still only has as much context as the subject matter expert was able to express. This can lead to costly assumptions because the programmer does not posses the full context that a subject matter expert has. In these scenarios a Business Natural Language can reduce the amount of overall work for a subject matter expert.

Using a Business Natural Language can also greatly simplify deployment of new business logic. For example, in the last application I delivered the business logic was stored in a database. The application provided a UI that allowed the subject matter experts to make changes to the business logic. When a subject matter expert was content with their changes they would save their changes back to the database. In this specific application all changes are required to pass through a workflow before becoming active. The entire workflow process is also done through the application's user interface. Submitted changes require approval from another employee before being included in the application. Once the logic is approved it may be immediately activated through the user interface. Each time a process executes it will execute all activated logic. In this scenario, the workflow is the deployment.

Program execution when using a Business Natural Language

A common concern when using a Business Natural Language is that the logic is executed in an expected way. For example, if you work for a casino and you want to find high rollers you could use these phrases to segment your player population:

put players who bet less than 100 dollars a hand into unimportant category
put players who bet between 100 and 150 dollars a hand into valuable category
put remaining players into high roller category

If the last phrase executes first, all of the casino's guests will be considered high rollers. Obviously, this is a serious problem. To mitigate this issue each phrase should be executable in isolation without dependency on any other phrase. The last phrase could be rewritten as such to conform to this rule:

put players who bet more than 150 dollars a hand into high roller category.

When statements can be executed in any order the dependency problem is solved; however, this does require that each statement contain all information required to execute the business logic. For this reason it is acceptable and even expected that duplication will exist in several if not all of your descriptive and maintainable phrases.

Along the same lines, it should be clear that you wouldn't chain or link multiple statements together. Also, a Business Natural Language would not contain variables or looping constructs. Again, each Business Natural Language phrase should be valid entirely in isolation.

Because of these limitations it should be clear that a Business Natural Language is not a silver bullet. For example, some companies use rules engines to manage their business logic. While a Business Natural Language could replace the simpler implementations, I do not expect that you would ever attempt to replace a complex implementation with a BNL. The value of a Business Natural Language can be derived almost directly from the narrowness of the scope of the problem being solved.

The constraints associated with Business Natural Languages , tracability becomes fairly simple. As the amount of logic increases the complexity does increase; however, since each logic phrase can execute in isolation it is fairly easy to follow behavior as the phrases execute individually. Also, at the heart of it a Business Natural Language is really a facade to an API. Therefore, if necessary, a programmer can step through each line of the Business Natural Language while as it executes.

Business Natural Language material

All of my recent focus has been on DSL work, specifically in the form of Business Natural Languages (BNLs). I considered writing a book on it and started putting together some material. The material may or may not end up in a book some day; however, for now I'm going to make it available here on my blog. In the writing it does make references to 'chapters' and some of the content is merely previous posts with new examples. You get what you pay for. =)

I hope you get something out of it, and feedback is welcome.

01 - Introduction
02 - The Problem
03 - The Solution
04 - DAMP BNLs
05 - Multiple Contexts

Redefine DateTime.now for consistent test results

Today I was looking at some code that used a class called Environment. The Environment class was used to specify a date based on what environment the code was executing in. In production the Environment class would simply return Datetime.now; however, when Environment was used in a test it always returned the value DateTime.new(2006, 1, 23, 13, 14, 15).

This can be particularly important if you are working with code similar to this:
class Comment
attr_reader :created_at

def initialize
@created_at = DateTime.now
end
end
In the example, you would like a test that proves that the created_at instance variable is set in the constructor, such as:
class CommentTest < Test::Unit::TestCase
def test_ctor_inits_created_at
assert_equal DateTime.now, Comment.new.created_at
end
end
Unfortunately, this test doesn't work correctly if the DateTime instances aren't created at exactly the same point in time.

To solve this problem the Environment class was designed to allow you to specify a date in the expected results of a test without being concerned if it matched the current date or not. The Environment solution worked, but it wasn't particularly elegant.

We decided to refactor to what we thought was a simpler solution. We added the following code directly to the test class.
class << DateTime
def now
DateTime.new(2006, 1, 23, 13, 14, 15)
end
end
By redefining the now method we were able to reliably assert equality. This change removed the need for the Environment class also. Of course, DateTime.now will always return this date for all the tests that follow this one; we were comfortable with all the tests depending on this constant value.

Sunday, July 30, 2006

Configuration File Demise

I'm not a big fan of predicting the future; therefore, this entry is more of a question than a statement. And, the question is, do dynamic languages mark the end of developer written configuration files?

Let's start with what's out of the scope of this conversation. Configuration files written or maintained by non-developers will not only live on but hopefully the maturity of Domain Specific Language development will deliver higher value configuration files.

This should leave us with the narrowly focused discussion of developer written and maintained configuration files.

Currently, most developer-centric configuration files are written in XML (though YAML has quickly caught on among those who dislike angle brackets). These configuration files often allow you to transform strings into objects or graphs of objects. Clearly, the abstraction required to make this work can be custom developed or you can make use of an existing library such as YAML.rb or the .net configuration classes. However, these configuration frameworks increase overall application complexity by adding a dependency and requiring knowledge of the framework.

One notable benefit of using a configuration file is the ability to change application behavior without the need to recompile the application. However, dynamic languages do not require compilation. Therefore, you can change the behavior of the application simply by changing the class and deploying the changes. For example, in ruby, a file will be loaded repeatedly if you use load instead of require. For performance reasons require is generally used; however, if you had a file that you planned on changing without restarting the application you could use load instead.

Another notable benefit of configuration files is the reduction in noise within a configuration file. However, in dynamic languages, such as ruby, it is possible to use class methods and meta-programming to eliminate the unnecessary repetitive code. For example, Ruby on Rails uses a database.yml file to specify database connection information. However, you could just as easily use a class such as
class Development < DatabaseInformation
host "localhost"
username "jay"
password "password"
end
So, it appears it is possible to live in a world without developer-centric configuration files; however, is there any benefit to this shift? I believe there is. As I previously stated, each configuration framework requires adding a dependency and understanding of usage. By removing this dependency I can more quickly understand how the entire application works and I can focus on solving the problem at hand.

An (admittedly simple, yet real world) example is the configuration information for the Ruby Continuous Integration application my team recently wrote. The application needed to know the path to the directory where revision and build information was located. This information could be stored in a YAML file, but that would require using the YAML library to load the information. A much simpler solution was to simply create a class that stored the information:
class OutputDir
def self.builds
"/builds"
end

def self.revisions
"/revisions"
end
end
This implementation (obviously) allowed me to use OutputDir.builds when I needed to specify the builds directory. It is also very obvious exactly what is going on to any person who looks at this file. To me, this was the simplest thing that could possibly work.

The gain in simplicity is obvious; therefore, the real question is, are there any benefits that configuration files provide that simple classes cannot?

Quality Assurance

The current state of Quality Assurance is disappointing.

QA has become (or always was) a ritual that is performed, but the actual value of the act has been lost. Someone needs to (re?)define what QA means, based on the value it should provide. Then, from the definition of QA, we can determine what skills are needed to perform QA work.

Classic quality assurance, in my experience, includes a team ranging widely in skill levels. Some testers have vision, can write tests, find root causes for problems, etc. On the other end of the spectrum, I've seen testers who provide zero value. Their daily tasks include running the functional/acceptance tests written by someone else and that's it. The problem isn't the broad range of skills among testers, it's that there seems to be no minimum level of skills required. I have actually been told "I wasn't good enough to be a developer, so I decided to do QA." And, I've seen entire QA teams composed of people who simply didn't fit anywhere else. I completely disagree with this approach. Quality assurance is a job equally as challenging as development, analysis, etc and the people performing the testing need to be just as talented.

We should base our definition of QA on the value it can provide. Companies believe that the QA ritual must be completed before going to production; however, they rarely define QA tasks beyond stating that the application "must be tested." Therefore, I believe QA doesn't need to conform to the classic approach to satisfy those requirements. I would like to see the definition of QA activities include unit and functional test analysis to ensure proper coverage (including adding new tests where necessary), writing new acceptance tests with the business, and automating roll-outs to new environments. I'm sure there are more tasks that fit, but those are the ones that come to mind first.

In the past, development was revolutionized by adopting testing practices, it's possible that testing could achieve the same results by adopting some "development" practices. Because these tasks highly align with development tasks the teams could work very closely, learning and helping each other. This collaboration could produce higher quality standards in the future of software development.

Ruby Array and Hash frozen behavior

While working through a problem recently with Michael Schubert we noticed some interesting behavior.

We had an array of strings and one of the strings was unexpectedly being changed. We decided to use freeze to help us determine when the string was being changed. Unortunately, a frozen array of strings does not prevent the strings from being changed.
irb(main):006:0> strings = %w[foo bar baz]
=> ["foo", "bar", "baz"]
irb(main):007:0> strings.freeze
=> ["foo", "bar", "baz"]
irb(main):011:0> strings[0] << 'oo'
=> "foooo"
irb(main):012:0> strings
=> ["foooo", "bar", "baz"]
To solve this problem you should simply freeze the object being changed instead of the object that contains reference to the object being changed.
irb(main):013:0> strings = %w[foo bar baz]
=> ["foo", "bar", "baz"]
irb(main):014:0> strings[0].freeze
=> "foo"
irb(main):015:0> strings[0] << 'oo'
TypeError: can't modify frozen string
from (irb):15:in `<<'
from (irb):15
from :0
This behavior also applies to hashes
irb(main):016:0> hash = { :foo=>'bar' }
=> {:foo=>"bar"}
irb(main):017:0> hash.freeze
=> {:foo=>"bar"}
irb(main):018:0> hash[:foo] << 'baz'
=> "barbaz"
irb(main):019:0> hash[:foo].freeze
=> "barbaz"
irb(main):020:0> hash[:foo] << 'baz'
TypeError: can't modify frozen string
from (irb):20:in `<<'
from (irb):20
from :0
For related information on OpenStruct, check out this previous entry.

Saturday, July 22, 2006

Ruby eval with a binding

I recently discussed how you can evaluate blocks in different scopes by using instance_eval, yield, and block.call. In this entry I'll show how you can evaluate a string of ruby code in the scope of a block.

Lets start with a simple example:
class Foo
def create_block
Proc.new {}
end
end
In the example the Foo class has a method, create_block, that simply creates and returns a block. Despite the fact that this doesn't look very interesting, it actually provides you with the ability to evaluate any string of ruby code in the scope of the block. This gives you the power to evaluate ruby code in the context of an object without directly having reference to it.
proc = Foo.new.create_block

eval "self.class", proc.binding #=> Foo
When would you ever actually use such a thing? I recently used it on my current project while developing a DSL to represent SQL. Our current syntax looks like this:
Select[:column1, :column2].from[:table1, :table2].where do
equal table1.id, table2.table1_id
end
When the above code is evaluated the [] instance method (following from) saves each table name in an array. Then, when the where method is executed the method_missing method is called twice, once with :table1 and then with :table2. When method_missing is called we check that table name array includes the symbol argument to verify it is a valid table name. If the table name is valid we return an object that knows how to react to column names. However, if the table name is invalid we call super and raise NameError.

All of this works perfectly, until you start using sub-queries. For example, the following code will not work with the current implementation.
Delete.from[:table1].where do
exists(Select[:column2].from[:table2].where do
equal table1.column1, table2.column2
end)
end
Unfortunately, this needed to work, and using eval and specifying a binding was how we made it work. The trick is getting the table names array from the outer block into the inner block without explicitly passing it in somehow (which would have made the DSL ugly).

To solve this problem, within the where method we added:
tables.concat(eval("respond_to?(:tables) ? tables : []", block.binding))
Let's break this statement apart and see what it's doing. The first thing it's going to do is
eval "respond_to?(:tables) ? tables : []", block.binding
Which simply means "eval that statement within the scope of the block". In this case the scope of that block is:
Delete.from[:table1].where do .. end
This scope is a Delete instance, which does have a tables array (tables #=> [:table1]). Therefore, the statement will evaluate and return the tables array, and the rest of the statement could be pictured as this:
tables.concat([:table1])
This is simply going to concatenate all the table names into the tables array available in the inner block. After this one line change, the statement now produces expected results.
delete from table1 where exists (select column2 from table2 where table1.column1 = table2.column2)
For more info see eval in the core documentation.

Thursday, July 13, 2006

Pre-Commit Rake Task

Every time I'm going to commit code I try to follow the same steps:
  • Add unadded files to Subversion (svn add)
  • Delete missing files from Subversion (svn rm)
  • Update my local files from Subversion (svn up)
  • Run the tests (rake)
  • Display the status of the files (svn st)
Since I do this often, it makes sense to pull this into some kind of process:
namespace :svn do
task :st do
puts %x[svn st]
end

task :up do
puts %x[svn up]
end

task :add do
%x[svn st].split(/\n/).each do |line|
trimmed_line = line.delete('?').lstrip
if line[0,1] =~ /\?/
%x[svn add #{trimmed_line}]
puts %[added #{trimmed_line}]
end
end
end

task :delete do
%x[svn st].split(/\n/).each do |line|
trimmed_line = line.delete('!').lstrip
if line[0,1] =~ /\!/
%x[svn rm #{trimmed_line}]
puts %[removed #{trimmed_line}]
end
end
end
end

desc "Run before checking in"
task :pc => ['svn:add', 'svn:delete', 'svn:up', :default, 'svn:st']
If you add the above code to a .rake file in lib/tasks you can simply run rake pc when you are ready to commit. If everything executes successfully, you are ready to commit.

Obviously, this only works for Subversion, but you are using Subversion anyway, right?

Wednesday, July 12, 2006

Ruby Block Scope

While doing DSL work you constantly evaluate blocks (or code as strings) in various contexts. In fact, one key to using a DSL is the ability to evaluate in various contexts. Take this code for example:
class SqlGenerator
class << self
def evaluate(&script)
self.new.instance_eval(&script)
end
end

def multiply(arg)
"select #{arg}"
end

def two(arg=nil)
"2#{arg}"
end

def times(arg)
" * #{arg}"
end
end
The above code allows you to generate a SQL statement by calling the SqlGenerator.evaluate method with a block:
SqlGenerator.evaluate { multiply two times two }
=> "select 2 * 2"
However, you could also execute the same code in the context of a calculator class to receive a result:
class Calculator
class << self
def evaluate(&script)
self.new.instance_eval(&script)
end
end

def multiply(arg)
eval arg
end

def two(arg=nil)
"2#{arg}"
end

def times(arg)
" * #{arg}"
end
end
Which executes as:
Calculator.evaluate { multiply two times two }
=> 4
The (obvious) trick here is to use instance_eval to specify the scope in which the block executes. The instance_eval method evaluates either a string or block within the context of the receiver. The receivers in my examples were a new instance of SqlGenerator and a new instance of Calculator. When instance_eval is called on it's own, self is the receiver. Also, be sure to note that I call self.new.instance_eval. If I don't call the new method of self, the block will be evaluated by the class and not an instance of the class.

However, instance_eval is only one of the ways a block can be executed. Blocks can also be executed by calling the call method:
def go(&block)
block.call
end
go { 'called' }
=> "called"
In this case, the scope at creation time is used for evaluation.
class Foo
def bar
"bar"
end

def do_something(&block)
block.call
end
end
Foo.new.do_something { bar }
NameError: undefined local variable or method `bar' for main:Object
from (irb):18
from (irb):15:in `do_something'
from (irb):18
from :0
In the above example, even though bar is defined in Foo the block was created in the context of Object (in irb) and bar is not defined in object. If you change the code to be:
def bar
"this bar is called, but it's not the bar in Foo"
end
class Foo
def bar
"bar"
end

def do_something(&block)
block.call
end
end
Foo.new.do_something { bar }
=> "this bar is called, but it's not the bar defined in Foo"
Well, you see the results.

Another way to execute a block is by using the yield statement. The yield statement also executes using the context in which the block was defined.
def bar
"this bar is called, but it's not the bar in Foo"
end
class Foo
def bar
"bar"
end

def do_something
yield
end
end
Foo.new.do_something { bar }
=> "this bar is called, but it's not the bar defined in Foo"
Whether to use yield, block.call, or instance_eval all depends on the context in which you need the code evaluated.

Thursday, July 06, 2006

Ruby Continuous Integration

My new project is starting up and we needed to make a decision on what type of Continuous Integration we were going to use. On the previous project we used Bitten, but it constantly caused us problems. We were going to use DamageControl, but it doesn't quite appear to be ready for use. We even tried using the Continuous Builder plugin, but it causes Subversion to hang while it runs the build.

So, what's left? Rolling your own, of course. I'm not a big fan of re-inventing the wheel, but it seemed like my other options were still prototype or busted wheels.

We chose a fairly simple solution. We are using Subversion, so we added the following post-commit hook:
#!/bin/sh
sudo -H -u builder /bin/bash --login -c "cd /builds/commits && /usr/bin/touch $2"
This creates a file from the revision number passed as an argument to post-commit.

Then, in the /builds folder we have builder.rb
require File.expand_path(File.dirname(__FILE__) + "/builder_config")
require 'fileutils'
require 'rubygems'
require 'ruby-growl'

loop do
unbuilt = Dir["#{BuilderConfig.commits_folder}/*"].collect { |path| File.basename(path).to_i }.sort
exit if unbuilt.size == 0
latest_revision_number = unbuilt.pop
FileUtils.rm "#{BuilderConfig.commits_folder}/#{latest_revision_number}"
FileUtils.cd "#{BuilderConfig.source_folder}"
system "svn update -r#{latest_revision_number}"
FileUtils.mkdir_p "#{BuilderConfig.output_folder}/#{latest_revision_number}"
status = system("rake > #{BuilderConfig.output_folder}/#{latest_revision_number}/rake-output.log") ? :success : :failure
system "svn log -r#{latest_revision_number} -v > #{BuilderConfig.output_folder}/#{latest_revision_number}/svn-log.log"
system "touch #{BuilderConfig.output_folder}/#{latest_revision_number}/#{status.to_s}"
['10.0.1.1','10.0.1.2'].each do |ip|
devbox = Growl.new ip, 'Builder', ['success', 'failure'], nil, 'grrbabygrr'
devbox.notify status.to_s, "Build #{latest_revision_number} #{status.to_s}", ""
end
end
Builder.rb loops through the commits folder pulling all files that are in commits. When it finds a file it removes it and stores the latest_revision_number. It then updates the code to that revision, creates a directory for that revision in the output, runs rake and saves the output to the revision folder, saves the svn info to the revision output folder, saves a file called success or failure, and sends a Growl message to the pairing stations. We use cron to run builder.rb every minute. It's very low tech, but it seems to work well for us.

For displaying build results we created a rails project that's quite simple. All we needed was one controller to pull the file info from the /builds dir, one view to display the info, and one model to logically group data from each build.

Like I said, I'm not a big fan of rolling your own solutions, but it's nice when you are using tools that let you easily come up with a solution.