RSpec
I enjoy discovering when old, good ideas from the research community eventually trickle their way out into common practice, but sometimes what you discover surprises you.
For example, contracts are a great idea that should see wider use, especially in languages that provided assertions from day one. But after you discover the nth framework/tool for providing contracts in a language like C++ or Python that does not support inheritance or visibility, one gets a little deflated.
This week I accidentally came across RSpec from the Ruby community. It caught my eye initially because it is described as a "behavior-driven development tool" and one of its popular tutorials states on line one: "Behavior Driven Development is specifying how your application should work, rather than verifying that it works.", this sounds like a framework for me. Moreover, since contracts are at the core of behavioral interface specification languages (BISLs) like the Java Modeling Language (JML), and I'm "one of the JML guys," then I should get excited about RSpec, or at least learn something from it. But before I dig into RSpec, let's reflect upon Ruby a bit.
I learned Ruby when it had just "leaked" out of Japan many years ago. The only English documentation on the language then was a fragment of the API docs, and thus I had to learn the language by reading other peoples' code. I like Ruby. It purposefully or accidentally intelligently synthesizes some of the best ideas of Smalltalk and object-based languages with a prototype-based feel. By prototype-based feel I mean languages like JavaScript, Tcl and, I'm told, used in several of the scripting languages that I mentioned yesterday, namely Io, Logtalk, Lua, Omega, and REBOL. My experiences with these kinds of languages derives from the literature, namely Abadi and Cardelli's "Theory of Objects" and papers about the Self language. The fact that it gives you some access to its metaclass system, a la Smalltalk, in a relatively clean API unlike, say, the horrid APIs of Python and Perl, is also compelling.
Consequently, I have written a few thousand lines of Ruby, including some of the server-side processing for my research group's website, and thought it would be nice to see a clean OO scripting language like Ruby catch on (as it has, in spades).
So, I hear you say: "Hey, a simple OO language with a clean metaobject framework is ripe for the application or dependable software engineering principles, Joe!" I would say you were right, so lets see what has happened in the World of Ruby... so, back to RSpec.
The first thing to note is, while the 'B' in "BDD" means "behavior," it is not literally in the sense of BISLs, but instead is the "behavior" of the "Agile" community. *sigh* This already starts to worry me, but lets not throw the baby out with the bathwater, because sometimes riding on the coattails of a populist movement like "agile programming" (or "aspects" or "Java," for that matter!) is just a smart mechanism to effect change.
The API and common use of RSpec guides a developer down the path of connecting informal English sentences using modal verbs like "must" and "should" and code fragments which interpret the informal specification. Thus, "behavior" in this context is the informal, manual specification and linking of traditional requirements and hand-written unit tests.
Now, anyone familiar with my work in BON and verification-centric development will know that I think codifying requirements, domain analysis, and features in structured English is a Good Thing. And we have been developing a formal refinement between informal specifications in English and formal artifacts like requirements, concepts, tests, types, and assertions (look for a paper on this in 2010). So the juxtaposition of English and code is unsurprising to me.
The codification of assertions in the API is also interesting. Having methods like "should" and "should_not" are akin to jUnit methods like "assertTrue" and "assertFalse," though fit better with the vernacular of the domain. Permitting the definition of pre and postconditions of unit tests via "before" and "after" methods, akin to aspects and straight from the world of CLOS and MOP, is also nice to see. There is also integrated support for mock objects and the use of lambda expressions to talks about state in the pre-state of a method call is cute as well.
So in the end, I think RSpec is a pretty nice framework for specifying the behavior of Ruby code, but only if you are willing to accept the fundamental testing premise of agile programming: that hand-written unit tests should specify the behavior of a system. My criticisms of this approach are not unfamiliar. Hand-written executable tests are only maintainable at high-cost and are expensive to write early if one does not have (1) a fairly solid understanding of a domain and (2) pleasant customers who do not change requirements all of the time.
In other words, I am still unconvinced that in the key areas where agile programming is supposed to shine their fundamental tenant, that of test-driven development, holds true. If you are an agile practitioner and have evidence of this claim, please speak up!
I will write more on RSpec later this year after I get a chance to really take it for a test drive.
posted by Joe Kiniry at 10:08 AM
2 Comments:
Interesting post; I agreed with most of it. However, your assertion that unit tests are "expensive to write early" is only true if you're writing tests for the whole system at once, before coding any of it. But that's not how I or any developer I know does it.
Rather, the point of this style of development is that you write *one* small test, watch it fail, then make it pass and move on to the next, refactoring as you go. This way, you can have emergent design without sacrificing test coverage.
In other words, you only test the little bit you know about, then you move on to the next little bit. If the client changes the requirements, no problem -- you write a different test.
Hi @Marnen. Thanks for your post.
You'll note that my statement is not quite what you quoted.
It is instead, "Hand-written executable tests are only maintainable at high-cost and are expensive to write early if one does not have (1) a fairly solid understanding of a domain and (2) pleasant customers who do not change requirements all of the time."
The cost of one test is, roughly, the cost of authoring, tracing, and maintaining the test.
From what I gather from my research colleagues in the "agile" and "testing" subfields, objective analysis of this strategy has shown itself to be not any more cost effective than previous testing strategies, and much less cost effective than using declarative specifications.
Moreover, at least from personal experience, I have really tried both approaches for years and I can safely say that I never, ever want to write tests first and either throw them away or babysit them when requirements are changing. Instead, I'd much rather write a declarative spec that is short, focused, and to the point and encapsulates the behaviors of dozens or hundreds of tests.
That being said, having a test-first process at least guarantees that there are tests, which is better than most projects who schedule testing last and then cut the whole phase when schedules slip.
Post a Comment
<< Home