Glad to have you on board.
Welcome!
Good to see you here guys.
"Don't use VerifyAll. What!?!? That's right, half the time I use Verify(someMock) and the other half I don't even Verify anything (as I can verify the result of a method called). Your tests should be complete enough that it shouldn't matter if ObjectA called ObjectB as long as ObjectA gives you the correct result."
Was just re-reading this part of the other article, does this not technically mean that you prefer state testing with stubs?
I guess I just don't consider it a mock if I don't verify, but I'm guessing you'd use mocks (even ones that aren't verified) instead of real components which is why you're not a classicist.
Anyway found your points at the end of the other post very relevant and I think if mockers followed your advice then mocking test quality would definitely rise.
@Colin
I think the whole "classicist vs. mockist" is a bunch of hooey. Baloney, malarkey even! Are there folks that don't use test doubles? I would say they're not doing real unit tests, but integration tests in disguise.
And, I think the "don't use VerifyAll" means "prefer SetupResult.For", and only do Expect.Call and Verify when you want it to break when it _isn't_ called.
@Aaron
That's what bugged me too the first time I saw it. Less mock methods, but now an enumeration that has...more ways to mock? How is that simpler?
I don't really find mock.Object simpler, either. That means I have to have a single Mock<T> instance for everything I Mock.
Could go on...
@Jimmy
Fowler makes it clear that classicists DO use doubles:
"The classical TDD style is to use real objects if possible and a double if it's awkward to use the real thing. So a classical TDDer would use a real warehouse and a double for the mail service. The kind of double doesn't really matter that much.
A mockist TDD practitioner, however, will always use a mock for any object with interesting behavior. In this case for both the warehouse and the mail service."
However using a test double is not the same as using mock. I also think that using mockist development does result in different results so I think Fowler is making a perfectly valid point.
"And, I think the "don't use VerifyAll" means "prefer SetupResult.For", and only do Expect.Call and Verify when you want it to break when it _isn't_ called."
I must admit I don't use Rhino Mocks much so I didn't immediately get this but I did Google it and I think I'm on the same page. I'm just thinking that if you setup mocks and don't verify any of their expectations then you've effectively created a stub (which is what SetupResult.For seems to be about).
This is obviously fine, just when I think of someone saying they're a mockist then I'm thinking they tend towards interaction testing over state testing. Rethinking this maybe that isn't a good assumption?
Agree that the "classicist vs. mockist" thing is obnoxious. You should just test things the way they make sense. The biggest laugh I have is when people hand roll mocks with boolean flags they set when something gets called, and then assert that's true... and then call it state based. Wow.
That said, there is value to not overmocking as it enables you to test things closer to the real world. There is also value to mocking everything because it allows tests to only test what they're trying to test... so I understand the argument.
Yeah, mostly stubs/setupresult like Jimmy said. I use Expect.Call when I expect it, and Expect.Call...Repeat.Never when it must not happen.
Pingback from Dew Drop - March 27 | Alvin Ashcraft's Morning Dew
sounds like a list of available pairs of Gap Jeans.
Just wanted to give you a heads-up that thanks to your (and others') feedback, we'll be simplifying the API to just Strict and Loose (code.google.com/.../detail).
Next version will no longer contain the documentation or intellisense for the additional (seldom used, we learned) members (they will be there just to redirect people via [Obsolete] to the appropriate replacement), and will be gone in the following version.
Thanks for your feedback! We need these sanity checks to keep the API as simple as possible.
Pingback from Reflective Perspective - Chris Alcock » The Morning Brew #65
What about jMock in the Java world. That doesn't use strings to identify methods but still has a "DSL" style API and can specify interactions between stateful objects, which Mockito cannot.
Johnson,
I haven't looked at it to be honest. I'll check it out.
Proposing to throw exceptions for normal program flow violates the .NET framework design guidelines and is typically a big no-no: blogs.msdn.com/.../396787.aspx
"Do not use exceptions for normal flow of control. Except for system failures, there should generally be a way to write code that avoids exceptions being thrown. "
See also msdn2.microsoft.com/.../ms229030.aspx
That's why I chose a SendMail example. Not being able to send a mail can be considered a system failure and is not normal flow. But you're right, generally you should not be throwing exceptions for normal flow, I was not recommending that and I should have clarified that.
In Java you can also use Ruby's Mocha framework (on JRuby) see for example
memeagora.blogspot.com/.../mocking-jruby.html
Arnon
Pingback from Dew Drop - April 9, 2008 | Alvin Ashcraft's Morning Dew
I really like the rails vim plugin. After I started using that was the first time I realized you could script vim using ruby, although I haven't had much reason to do it yet.
I've been playing around with textmate for rails and even with all those scripts I keep finding myself wanting to go back to vim.
I think the only thing that really bothers me about vim is the lack of a good file browser. I use minibufexpl but it isn't near textmates drawer (i think that's what it's called)
Pingback from Dew Drop - April 13, 2008 | Alvin Ashcraft's Morning Dew
Pingback from Reflective Perspective - Chris Alcock » The Morning Brew #73
I don't know about providing events on mocked objects, but I use a pretty simple class to ensure that some code under test does indeed call an event. I've written about it here: www.exotribe.com/.../36
Pingback from is there a difference between asking and telling? « monkey island
I'm not normally one that gets excited by new syntactic sugar but I saw the example you put on twitter and immediately liked it. Very cool.
Best thing was I looked at it and thought "It", what’s that about? I was expecting it to be some mad work around but when I downloaded the Machine codebase I found it was really elegant and understandable. Made me feel dumb (in a good way).
Anyway to me the "=()=>" is not a major issue, it’s at the end of the line and easily ignored. The static field thing threw me a little but I guess with the BDD style of small focussed fixtures it won't be an issue.
Only thing that I'm still unsure of is whether just having [Description] is that useful, would I not also want to use something like this group my specifications (especially given that with the context/specification approach I might end up with lots of fixtures for one higher level behavior such as funds transfer). Guess it depends on how you use these specifications but finding ways to group them seems to be important.
My first impression is also that you are not necessarily covering exactly the same area as NBehave, my feeling it is xBehave is _supposed_ to be most suitable for higher level story tests where as to me the xSpec approach is more generally applicable.
Nice work man!
Pingback from Dew Drop - May 8, 2008 | Alvin Ashcraft's Morning Dew
Neat!
The =()=> notation with fields is a clever way to work around the fact that we can't otherwise define new "shapes" for methods.
hmmm... I can see interesting things coming out of this...
This is brilliant, Aaron.
Does this contribute to the 2 articles you have to write about liking TypeMock for the free license? :)
That's pretty cool.
What about NMock 2? Did you test on that?
I don't really consider NMock2 a framework worth considering, as it lacks a strongly typed, refactor friendly syntax, so no, I didn't. Feel free to add a Strategy though and give it a shot.
Just wanted to quickly note that I tracked down the performance issue in Rhino.Mocks and patched it
And I was waiting till this morning to comment that your post would be like a red rag to a bull, and I fully expected Oren to drop all other work until Rhino was ten times faster than the others :)
Apart from that flipant comment, nice work!!!
Pingback from Reflective Perspective - Chris Alcock » The Morning Brew #90
Cheers mate! Well done
Pingback from Dew Drop - May 9, 2008 | Alvin Ashcraft's Morning Dew
This is very awesome. However...
While I understand your choice for doing it in C#, I feel that by not using a language that would allow for a more fluent DSL syntax, you have created something that will hamper it's adoption.
The way you are using C# goes against the grain of how users are taught to use C#. static fields and what look like fancy function but end in semicolons. If I brought this to my team it'd be very confusing for them because it *is* C# but it's not the C# they're use to.
That could be just dumped into the "learning curve" bucket, but I see it being a recurring problem with the context switching required between MSpec and C# when bouncing between tests and code.
It could be argued that the context switching between MPsech C# and "regular" C# when bouncing back and forth between tests and code is no more grating than if it were a completely separate DSL, but I feel that all the syntactic sugar required to make this work prevents me from forgetting that it is C#. There is a certain amount of muscle memory in "private Account fromAccount;"
Understanding Model-View-Controller : Introducing developers to the MVC concept can, at times, be a challenging endeavor. Jeff Atwood has put together a very informative post that may be able to help! Mocking Frameworks Benchmarked : In search of a faster
I was equally confused when looking at a performance trace of Moq vs. Rhino. (I don't have TypeMock installed.) The majority of the time in both Moq and Rhino was spent creating proxies and both frameworks are using DynamicProxy2 for proxy creation. Something was definitely up. Excellent work tracking down the problem!
James,
I hear you. It's definitely a stretch of C#. The benefits of using C# are clear though. The tool and the framework support make it worth it in my opinion. Like I said in the post though, this is an experiment. I've got no ego behind this and I'm not pushing for widespread adoption. I'm just kind of dropping it on the community and seeing if it sticks at all :)
Last week I posted about some troubles I was having with the unit testing frameworks for F#. Today, Brad
Pingback from The Unit Testing Story in F# Revisited | Developer Home
Rhino.Mocks Beta Bits are Running Turbo-Charged!
Hi Aaron,
You might have already mentioned that somewhere but I can't seem to find it.. How do you manage to have a pretty print output after you run your test in TestDriven.NET. ?
Thanks,
Take care
+1
I like context setups, too. Picking a good name for the context provides as much value as a well-named helper-method.
Michael,
TD.NET allows you to print whatever you like when implementing a custom runner. I just used that.
It's not super flexible (errors always print the same, along w/ the stack trace) so I save the errors until the end in TD.NET.
Balancing DRY against readability definitely seems more important in BDD with the small fixtures.
Your last comment about helper/base classes also resonates with me, I've been using base classes a bit and although the inheritance is only one level deep there is no doubt they hurt readability.
Looks awesome man, I'll grab it and run it through its paces this week! I'm glad you guys (you and Scott) put your heads together. Thanks!
You've been kicked (a good thing) - Trackback from DotNetKicks.com
It's been a while, but we've gotten several new things into Machine.Specifications (MSpec). I'm
Pingback from Reflective Perspective - Chris Alcock » The Morning Brew #171
This is really awesome. I will have put it through it's paces.
Pingback from Dew Drop - September 3, 2008 | Alvin Ashcraft's Morning Dew
Pingback from MSpec v0.2 - taccato! trend tracker, cool hunting, new business ideas
Good post but I think that the best way to ensure that the specs are written in the users language (or the UL) is to get them involved. If you do that then words like "invoking" only make it into the specs (GWT or not) if your user/domain expert is happy with the term.
Colin,
I completely agree that they should be involved. The level of involvement can vary from shop to shop though, and I certainly don't expect my users or BA's to write all of the specs with me. Their job in our shop is to aid in writing the stories and to answer language or content questions we may have while writing specs. I think the key is to get developers thinking like them and to just be aware of the fact that they are not writing specs for other developers to understand necessarily.
From there you do what you need to do when writing the specs whether it be involve your user/BA/customer in every one or just certain areas + review.
Yeah I hear what you are saying, and I didn't mean to infer they should be involved in all specs.
What I should have said was that I don't think GWT hampers BDD because Dan North was primarily pushing it for specs that did come out of work with BAs etc (which I agree is just a subset of the specs you write).
Here's a question for you though, have you found reporting on all of your specs is really useful and are your users/BAs/domain experts really interested?
I can see that the reports on the story based tests have value but you then write a lot of detailed/technical specifications (mapping, process, controller, integration) and I haven't seen any value in exposing that level of detail outside the dev team. I guess it could have value for certain types of specs though, particularly those against the domain itself?
Not suggesting writing your specs in developer language though...
Pingback from Arjan`s World » LINKBLOG for October 19, 2008
I feel that behavior testing is more interesting than scenario testing. Obviously you need scenarios to test the behavior, but I feel it is important to name those scenarios appropriately. There is a large difference between "Given an account with $5, When I withdraw $6" and "When withdrawing more money than an account has"
GWT/Scenario based frameworks have this concept baked in. The whole notion of turning the GWTs into methods with parameters is awkward and forces you to concentrate on the scenario when you should really be looking at and documenting the behavior. Context/Specification, when applied correctly, does just this.
To be perfectly honest, we've only recently started making heavy use of MSpec and creating specs. We haven't yet begun tagging the specifications that would be useful to BAs etc and pulling them out to show them. We do plan on it, but it hasn't happened yet. I'll definitely describe my experiences when we do. I'm quite confident that when we do, the ones written in their language will make more sense to them.
And yes, the lower level specs are more technical in nature, but care still needs to be taken to focus on the behavior and not the implementation. It's subtle, but important.
Domain specs are a set of specs that I think are very important to keep pure and in as much UL as possible. This is where much is born, and it is important to communicate with the customer/ba/etc while doing this. This is something we have been doing and it has led to greater insight into our domain.
Also, I'd be careful calling tests "story based". Stories are transient statements of feature to business value mapping. Though working to deliver them does drive the creation of specifications, I would shy away from tying them to the stories in any permanent capacity.
On the behavior vs scenario thing, even when I do use GWT I would usually use the second style. So I might phrase it as "Given an account that is in credit, when I withdraw more money than the account has in it". Actually thats the style of scenario that Dan North originall described (dannorth.net/introducing-bdd), though I guess things might have moved on since.
On the reporting. I evaluted MSpec last week and really loved it. Great idea, great work. However in the end we decided not to use it (for now at least). For me one big loss from not using it might be the reporting so I'll be interested to see what you find.
My thinking is that the specs that are valuable to the BA or whoever are ones that we involve them in upfront, and with that in mind the reporting *might* have limited value. But since I haven't tried I don't really know...
On transitive nature of stories, I see what you are saying. I'm only tying them to stories because the stories drive them out and for the upper level tests, the ones that most interest the BAs/domain experts etc, the wording matches closely.
You're correct that Dan North's initial introduction used more sensible naming and was less focused on the specifics of the scenario. Unfortunately, the community has taken it another way w/ many of the GWT frameworks that have become focused on DRY and "plug in the values" testing.
Despite this, I still feel the C/S method is more readable. It can also lead to more important abstractions. It's very easy to tie multiple Givens together and not realize that what you're describing actually has an important name.
The reporting is valuable for review. We've already seen value in that. I'll look over new specs written by the team or we'll review them as a team. It's an excellent way to catch things that just don't behave the way they should or the wording is awkward or incorrect. You can do the same in the code, but having the report removes even more of the noise than MSpec ever could.
I agree, plugin the values doesn't make too much sense to me. What I do find useful is being able to reuse fixtures/contexts, and parts of them, when doing acceptance testing though. I say this because you end up with just slight variants of the same thing. For example what happens if X is valid and Y isn't, then swap. Or we've got very similar specs other than the type of user that's logged in or whether they logged in using one mechanism or another.
It was those cases that I'd hoped the xBehave frameworks would help with but I didn't like the result so I just fell back on writing my own classes/base classes/helper classes. Not perfect but doable. Still my feeling is composition is key at acceptance test level. In fact I really need to look again at JBehave as I think its taking it way beyond what we have in .NET.
On readability, not really sure one way or the other. I find the GWT syntax works quite well especially at the acceptance level where there is a lot to the given. To be honest though I think I could dump GWT and use C/S for all specs, but at the moment I haven't found a pressing reason to go down that path.
Missing abstractions, that sounds interesting. Got an example I can mull over?
The reporting being valuable for review within the team, hadn't thought of that and yeah I can see the advantage of that especially in a code review setting, a lot nicer than manually going through and seeing what tests were written.
On the "subtle difference" thing, "should" has started to really annoy me... I prefer "Message Board, when viewed, shows only the five most recent messages".
Mmm.... Aaron, when I said that was the most twisted C# I'd ever seen, I understated just how cool it is :) Still in awe of this framework!
Pingback from Dew Drop – October 20, 2008 | Alvin Ashcraft's Morning Dew
I don't understand why seeing private declaration is deemed noisy or why it's a problem to see the indentation tab stopped at the same column as of the previous line. Writing private would definitely make your intentions clear.
Just what I was looking for... Thanks for posting it. I would love one more showing some of the DRY stuff we talked about.
Rohit,
The language makes your intentions clear. Private is an unnecessary keyword. Any unnecessary ceremony is just that. As such, it should be treated lightly and removed because it does not add value.
Furthermore, these tips are for the MSpec framework. A framework which has no understanding of "private" so it's even more superfluous. The entire point of the framework is to remove language noise and focus on the essence of the specs.
With regards to the indentation, do you indent your method bodies to begin under the ()'s? In MSpec, these initializers are essentially methods, just reshaped. Removing the excess indentation treats them as such.
Dave,
"Should" is intended to communicate always and pervasively that specifications are soft and changeable. It's a word that sets up a psychological context that encourages flexibility and challenges the rigidity that sets in when specifying behaviors with finalistic language.
Pingback from Arjan`s World » LINKBLOG for October 20, 2008
Scott,
"Should" seems to me to be unnecessary noise when reading through the generated specs, especially in a system that has been in active development for a long period.
Increasingly, as time goes on, a vast majority of the specifications refer to functionality that has already been implemented. I prefer to think of the specs as a description of the system's behavior, rather than a set of assertions.
As to "should" clarifying the malleability of requirements, I sort of get what you're saying, I just don't buy it. It's semantic hair splitting in my opinion.
Also, I have a tendency to document limitations and assumptions in my code with tests (descriptions of what the code actually does, in it's current state)..
In these cases, it's not really a matter of "it should do X" but more like "it does X because X is all we need for now"
Nice tip if you only code specs. I don't want the rest of my code look like garbage.
Sergey,
Wow. Garbage? Not having superfluous privates and indenting sanely is garbage? If you say so :)
What's suppress warning 169 for? What is 169?
169 is unused private. It's what makes the instance fields (It, Because, Establish, etc) gray, and it can give you a warning when building.
Glad you found those posts helpful. Thanks Aaron!
I’ve been reading quite a bit lately about setters and encapsulation , but for whatever reason, it never
Wow. That's harsh...why not just drop it for one of the many open source options out there?
It's free and no sales people to deal with?
Simon....
That's a sad tale. I've always respected ThoughtWorks but this sounds awful!
A possible explanation for this is that you're not the type of customer they want to attract. It might be that they their target customers are big companies with large teams with huge budgets for licensing, support and training.
Simon,
Like I said, it's currently the closest match for our needs. The oss offerings are lacking. I don't mind paying for it as long as the pricing is reasonable.
Tobin,
Yep, I think their target customer is the people they consult for (oh and by the way, you'll absolutely *love* our PM tool, we'll just add it on the bill!) and like you said, large companies that don't care about money.
I'm not sure there is much value complaining about pricing. The essence of your concern about the pricing is that Mingle is too expensive. If it is too expensive don't buy it.
any svn access for those of us who are not git users and don't want to learn a new source control system just for one project?
apparently i'm retarted and didn't see the "download" link on github. ignore my previous whining. :)
Cool. If you add a Readme.markdown file to your source tree? GitHub will display it very nicely on the project home page.
See this example I did this for Golem
github.com/.../master
Markdown also reads nicely in a text editor (so you can write it in Visual Studio).
Syntax rules are here, if you're interested
daringfireball.net/.../syntax
I like GitHub, nice to see another .NET project on there!
Liam,
Fair enough. I think my chief complaint stems from the pricing model, not necessarily the cost. But again, back to your point, if one cannot afford it, don't buy it.
Also, I've had a few friends I've tried to get to look into it but they won't even open the box because of the price tag. But hey, too expensive, don't buy it.
The significance of "should" is the impact that words like "does" and "must", etc, have on developer psychology.
The language can and often does create unnatural and false rigidity. More than an issue of splitting semantic hairs, it's an issue of reinforcing the right state of mind.
Commentary on the pricing is valuable and necessary because it's a matter of ethics.