Dan Miser - Unit testing with VS.NET 2008 Pro

Monday, February 04, 2008

Unit testing with VS.NET 2008 Pro

This link explains the unit testing tools available in VS.NET 2008 Professional. My take after reading that section is that Microsoft doesn't get it. Here is my list of observations about this section. Feel free to leave a comment to tell me where I'm wrong.

I feel like this is nothing more than an upsell to get me to go and buy VS.NET TeamSystem. Sorry, that's not going to happen. TeamSystem is way too expensive (both initially and in setup/maintenance time for a TFS server). Besides, we have an existing development infrastructure setup since we use multiple (non-MS) languages/platforms here.
It looks like MS wants to ignore the thriving and long-standing community of open source unit testing tools. I'm willing to bet that NUnit (and MbUnit, etc.) has more code using it than Microsoft.VisualStudio.TestTools.UnitTesting. Yet, if I want to use the integrated Test menu in VS.NET, I need to rewrite my unit test code to use near-identical attributes for my unit tests (e.g. [TestMethod()] vs. [Test]).
Code coverage is not available in VS.NET Pro. I guess code coverage isn't something that developers should be doing - well, unless they shell out tons of money to get it done. Thankfully, there are free alternatives, with cheap upgrades to the latest version.
I want to easily run and debug my unit tests. I know Resharper can do this, but I haven't installed that in VS.NET 2008 due to the number of issues I had with it. (Yes, I know an EAP is coming out soon, but I also have CodeRush and Refactor, which work fine with 2008.) I like TestDriven.net, but I'd like to be sure that it doesn't go away.
The documentation is too cluttered. If you're going to give me a list of what's included and what's not included in the Pro version, then why not just separate the documentation to make it easier for me to see what capabilities I have (and what I don't).

In short, it's disappointing that MS has chosen to force people to one specific way of writing unit tests that is against industry norms. ASP.NET has shown me that they do understand the provider model, so why not use it here to allow me to run and debug my unit tests using whatever tools I want.

Monday, February 04, 2008 11:01:31 AM (Central Standard Time, UTC-06:00)

Comments [5] -
.NET

Wednesday, February 06, 2008 11:09:40 AM (Central Standard Time, UTC-06:00)

Dan, I wish I could argue with you, but no luck. The unit test tools in VS2008 really are a least common denominator solution. I still recommend that people start with them because they are built in and supported. As people become more facile with Unit Tests/TDD I think they will naturally look for better solutions.

++Alan

Alan Stevens

Wednesday, February 06, 2008 11:16:25 AM (Central Standard Time, UTC-06:00)

The first question I get asked when demo'ing TDD at work is why I'm not using VSTest. Most don't seem to believe that unit testing existed before MS added it to the IDE...and don't get me started on NAnt vs. MSBuild. Great post and I feel your pain.

Erik Lane

Wednesday, February 06, 2008 11:49:14 AM (Central Standard Time, UTC-06:00)

Wow! Your post reeks of anger, like someone put a gun to your head to evaluate TeamSystem and then waterboarded you to get a recommendation. All your comments spring from an assumption that Microsoft is forcing their crappy solution on you, then you rant about all the free and effective alternatives that you love. Yet none of the few alternatives you mention do everything you need, many are tightly focused in one area. Why hold a unified solution up to those things?

I also have a lot of reservations about Team System. Mainly that few organizations have the discipline to use it effectively (even assuming the budget is there) and thus definitely they will break down if they try to manage the nearly infinite combinations of "great open source and free" tools combined with multiple paid tools like Resharper and the attendent versioning management (of the tools, not your products). I just heard Scott Hanselman gripe about getting blindsided by Log4Net, a variety of tools can bring their own headaches no matter who produces them.

Walter Lounsbery

Wednesday, February 06, 2008 12:04:28 PM (Central Standard Time, UTC-06:00)

I think anger is valid when Microsoft believes so fully in the NIH (not invented here) strategy. In other languages and other IDEs they don't invent their own unit testing frameworks, they integrate with existing ones. Often more than one. See in the Java world with JUnit and TestNG for example.

Where is the Mock framework in the MS Solution...there isn't one. Where is the code coverage solution...there isn't one. If you're not using Mock Objects, you're not unit testing because you're not isolating your classes from their dependencies (and your design probably sucks to boot from tight coupling).

NUnit has a reasonable Mocking solution, it works with NCover. A real IDE would support these tools out of the box.

Geoff Lane

Wednesday, February 06, 2008 12:36:05 PM (Central Standard Time, UTC-06:00)

Walter,

First off, I am *disappointed* about MS's inability to "get it" with respect to agile methodologies. It would not have been hard to integrate existing unit testing open-source tools like TestDriven.net did. It would not be hard to add full support for things like code coverage without feeling the need to reinvent a whole set of tools, and doing it half as well, and only baking them fully into a bloated, over-priced version of VS.

Secondly, my post is doing exactly what a blog should: eliciting conversation. I started the conversation with my observations and experiences. You continued the conversation, which is good. Of course, I would have preferred it had you actually rebutted something that I said, as opposed to just contradicting me. But still, some conversation is better than none.

Dan Miser

Comments are closed.