Discussion:
Another Question of Naming
Charlie Poole
2008-09-08 23:57:24 UTC
Permalink
Hi All,

I'm still not happy with one of the attribute names we are
using for parameterized tests, so I'm asking for ideas,
opinions and suggestions.

Here's what we have now...

You can specify entire test cases together or individual
values for parameters, which NUnit then combines into
test cases using various strategies.

If you go with full test cases, they can either be inline
or separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate

Similarly, if you go with individual parameters, they
can be inline or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate

We'll be adding others - cases in a spreadsheet, etc. - but
these are the four basic ones.

I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the
same.

But DataSource strikes me as a bit too generic. It
doesn't tell me that it's a set of single parm values
rather than a test case. It sounds like it might have
something to do with a DB. But for the life of me
I can't think of anything better.

Ideas?

Charlie




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Olof Bjarnason
2008-09-09 06:46:33 UTC
Permalink
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for parameterized tests, so I'm asking for ideas,
opinions and suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for parameters, which NUnit then combines into
test cases using various strategies.
If you go with full test cases, they can either be inline
or separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they
can be inline or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
I'm not 100% how this "paramater values" version of test cases work.

I understand TestCase and TestCases (even though TestCases is a bit
ambious to me since it actually specifies a named source of
TestCase-tuples, but it's so readable anyway)

Could you give an example of the Values, DataSource including a
[Test]-method (or TestCase method if that is the way to use them..)?
Post by Charlie Poole
We'll be adding others - cases in a spreadsheet, etc. - but
these are the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the
same.
But DataSource strikes me as a bit too generic. It
doesn't tell me that it's a set of single parm values
rather than a test case. It sounds like it might have
something to do with a DB. But for the life of me
I can't think of anything better.
Ideas?
Charlie
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "NUnitV3" group.
For more options, visit this group at http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-09 12:49:49 UTC
Permalink
Hi Olof,
Post by Charlie Poole
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for
Post by Charlie Poole
parameterized tests, so I'm asking for ideas, opinions and
suggestions.
Here's what we have now...
You can specify entire test cases together or individual values for
parameters, which NUnit then combines into test cases using various
strategies.
If you go with full test cases, they can either be inline or
separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they can
be inline or
Post by Charlie Poole
separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
I'm not 100% how this "paramater values" version of test cases work.
I understand TestCase and TestCases (even though TestCases is
a bit ambious to me since it actually specifies a named
source of TestCase-tuples, but it's so readable anyway)
Yes, I considered TestCaseSource, which is literally more exact,
but I liked TestCases with the notion being "The test cases are
to be found here"
Post by Charlie Poole
Could you give an example of the Values, DataSource including
a [Test]-method (or TestCase method if that is the way to use them..)?
Here's one - a bit contrived, but illustrative...

[Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[Values(1000, 2000)]int b)
{
Assert.That( a+b == b+a );
}

Since the default strategy is combinatorial, this would
generate 8 test cases.

The equilent with datasource would use
[DataSource("xxxxx")] and [DataSource("yyyyy")]
for the attributes, were xxxxx and yyyyy would
be declared elsewhere in the class, probably
as arrays of ints.

Specifying parameter values separately is a key
to more advanced techniques like Pariwise and
Combinatorial testing and Theories.

I've also used it with a Sequential strategy
(just use the args in sequence) when I needed
to pass argument 1 to method 1 and arguments
1 and 2 to method 2.

Charlie
Post by Charlie Poole
Post by Charlie Poole
We'll be adding others - cases in a spreadsheet, etc. - but
these are
Post by Charlie Poole
the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the same.
But DataSource strikes me as a bit too generic. It doesn't tell me
that it's a set of single parm values rather than a test case. It
sounds like it might have something to do with a DB. But
for the life
Post by Charlie Poole
of me I can't think of anything better.
Ideas?
Charlie
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "NUnitV3" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-09 12:57:40 UTC
Permalink
Those are possibilities...

Charlie


_____

From: cliff vaughn [mailto:***@gmail.com]
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Cc: nunit-***@lists.sourceforge.net; ***@googlegroups.com
Subject: Re: [nunit-developer] Another Question of Naming


What about ValueSource or ValuesSource?


On Mon, Sep 8, 2008 at 6:57 PM, Charlie Poole <***@nunit.com> wrote:


Hi All,

I'm still not happy with one of the attribute names we are
using for parameterized tests, so I'm asking for ideas,
opinions and suggestions.

Here's what we have now...

You can specify entire test cases together or individual
values for parameters, which NUnit then combines into
test cases using various strategies.

If you go with full test cases, they can either be inline
or separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate

Similarly, if you go with individual parameters, they
can be inline or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate

We'll be adding others - cases in a spreadsheet, etc. - but
these are the four basic ones.

I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the
same.

But DataSource strikes me as a bit too generic. It
doesn't tell me that it's a set of single parm values
rather than a test case. It sounds like it might have
something to do with a DB. But for the life of me
I can't think of anything better.

Ideas?

Charlie




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100
<http://moblin-contest.org/redirect.php?banner_id=100&url=/> &url=/
_______________________________________________
nunit-developer mailing list
nunit-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks

cliff
Olof Bjarnason
2008-09-09 13:32:23 UTC
Permalink
OK I understand the ValuesAttribute / DataSourceAttribute now.

And I agree with cliff that Value[s]Source is more readable than DataSource:

Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[ValueSource("b-values")]int b)
{
Assert.That( a+b == b+a );
}
Post by Charlie Poole
Those are possibilities...
Charlie
________________________________
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Subject: Re: [nunit-developer] Another Question of Naming
What about ValueSource or ValuesSource?
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for parameterized tests, so I'm asking for ideas,
opinions and suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for parameters, which NUnit then combines into
test cases using various strategies.
If you go with full test cases, they can either be inline
or separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they
can be inline or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
We'll be adding others - cases in a spreadsheet, etc. - but
these are the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the
same.
But DataSource strikes me as a bit too generic. It
doesn't tell me that it's a set of single parm values
rather than a test case. It sounds like it might have
something to do with a DB. But for the life of me
I can't think of anything better.
Ideas?
Charlie
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks
cliff
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"NUnitV3" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-09 13:38:26 UTC
Permalink
Hi Olof,

I'll wait a bit to see if any other ideas pop up, but
it makes sense.

BTW, internally I use "source" to mean something that
gives us data - either test cases or individual args.

I've settled on "provider" as the next level of
indirection: a provider -> source -> data.

So "source" is a good word here.

Charlie
-----Original Message-----
Sent: Tuesday, September 09, 2008 6:32 AM
Subject: [nunitv3] Re: [nunit-developer] Another Question of Naming
OK I understand the ValuesAttribute / DataSourceAttribute now.
And I agree with cliff that Value[s]Source is more readable
Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[ValueSource("b-values")]int b)
{
Assert.That( a+b == b+a );
}
Post by Charlie Poole
Those are possibilities...
Charlie
________________________________
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Subject: Re: [nunit-developer] Another Question of Naming
What about ValueSource or ValuesSource?
On Mon, Sep 8, 2008 at 6:57 PM, Charlie Poole
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for
Post by Charlie Poole
Post by Charlie Poole
parameterized tests, so I'm asking for ideas, opinions and
suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for
Post by Charlie Poole
Post by Charlie Poole
parameters, which NUnit then combines into test cases
using various
Post by Charlie Poole
Post by Charlie Poole
strategies.
If you go with full test cases, they can either be inline or
separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they can
be inline
Post by Charlie Poole
Post by Charlie Poole
or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
We'll be adding others - cases in a spreadsheet, etc. -
but these are
Post by Charlie Poole
Post by Charlie Poole
the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the same.
But DataSource strikes me as a bit too generic. It doesn't tell me
that it's a set of single parm values rather than a test case. It
sounds like it might have something to do with a DB. But
for the life
Post by Charlie Poole
Post by Charlie Poole
of me I can't think of anything better.
Ideas?
Charlie
---------------------------------------------------------------------
Post by Charlie Poole
Post by Charlie Poole
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Charlie Poole
Post by Charlie Poole
Moblin SDK & win great prizes Grand prize is a trip for two to an
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks
cliff
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the
Google Groups "NUnitV3" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-12 04:01:50 UTC
Permalink
I just thought of another one: ParameterSource?

Charlie
-----Original Message-----
Behalf Of Olof Bjarnason
Sent: Tuesday, September 09, 2008 6:32 AM
Subject: Re: [nunit-developer] [nunitv3] RE: Another Question
of Naming
OK I understand the ValuesAttribute / DataSourceAttribute now.
And I agree with cliff that Value[s]Source is more readable
Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[ValueSource("b-values")]int b)
{
Assert.That( a+b == b+a );
}
Post by Charlie Poole
Those are possibilities...
Charlie
________________________________
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Subject: Re: [nunit-developer] Another Question of Naming
What about ValueSource or ValuesSource?
On Mon, Sep 8, 2008 at 6:57 PM, Charlie Poole
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for
Post by Charlie Poole
Post by Charlie Poole
parameterized tests, so I'm asking for ideas, opinions and
suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for
Post by Charlie Poole
Post by Charlie Poole
parameters, which NUnit then combines into test cases
using various
Post by Charlie Poole
Post by Charlie Poole
strategies.
If you go with full test cases, they can either be inline or
separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they can
be inline
Post by Charlie Poole
Post by Charlie Poole
or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
We'll be adding others - cases in a spreadsheet, etc. -
but these are
Post by Charlie Poole
Post by Charlie Poole
the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the same.
But DataSource strikes me as a bit too generic. It doesn't tell me
that it's a set of single parm values rather than a test case. It
sounds like it might have something to do with a DB. But
for the life
Post by Charlie Poole
Post by Charlie Poole
of me I can't think of anything better.
Ideas?
Charlie
---------------------------------------------------------------------
Post by Charlie Poole
Post by Charlie Poole
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Charlie Poole
Post by Charlie Poole
Moblin SDK & win great prizes Grand prize is a trip for two to an
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks
cliff
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "NUnitV3" group.
unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Olof Bjarnason
2008-09-12 05:53:59 UTC
Permalink
Post by Charlie Poole
I just thought of another one: ParameterSource?
So we started with these four names:

TestCase( object, object, ... )
TestCases( string )
Values( object, object, ... )
DataSource( string )

.. and now we are here:

TestCase
TestCases
Values
ParameterSource

I liked ValueSource( string ) better than ParameterSource since it
clings better to Values, so it is more analogous to the TestCase,
TestCases pair.

But then again the only thing differentiating TestCase and TestCases
is singular/plural form.

I guess the hurdle is that the concept of a "named source" is used in
both pairs, but the name "Source" only in one. It is a bit
inconsequent. Either both should use Source, or none of them:

1.
TestCase
TestCaseSource
Values
ValuesSource

2.
TestCase
TestCases
Values
ValuesLookup (or something ..)
Post by Charlie Poole
Charlie
-----Original Message-----
Behalf Of Olof Bjarnason
Sent: Tuesday, September 09, 2008 6:32 AM
Subject: Re: [nunit-developer] [nunitv3] RE: Another Question
of Naming
OK I understand the ValuesAttribute / DataSourceAttribute now.
And I agree with cliff that Value[s]Source is more readable
Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[ValueSource("b-values")]int b)
{
Assert.That( a+b == b+a );
}
Post by Charlie Poole
Those are possibilities...
Charlie
________________________________
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Subject: Re: [nunit-developer] Another Question of Naming
What about ValueSource or ValuesSource?
On Mon, Sep 8, 2008 at 6:57 PM, Charlie Poole
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for
Post by Charlie Poole
Post by Charlie Poole
parameterized tests, so I'm asking for ideas, opinions and
suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for
Post by Charlie Poole
Post by Charlie Poole
parameters, which NUnit then combines into test cases
using various
Post by Charlie Poole
Post by Charlie Poole
strategies.
If you go with full test cases, they can either be inline or
separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they can
be inline
Post by Charlie Poole
Post by Charlie Poole
or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
We'll be adding others - cases in a spreadsheet, etc. -
but these are
Post by Charlie Poole
Post by Charlie Poole
the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the same.
But DataSource strikes me as a bit too generic. It doesn't tell me
that it's a set of single parm values rather than a test case. It
sounds like it might have something to do with a DB. But
for the life
Post by Charlie Poole
Post by Charlie Poole
of me I can't think of anything better.
Ideas?
Charlie
---------------------------------------------------------------------
Post by Charlie Poole
Post by Charlie Poole
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Charlie Poole
Post by Charlie Poole
Moblin SDK & win great prizes Grand prize is a trip for two to an
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks
cliff
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "NUnitV3" group.
unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-12 18:34:46 UTC
Permalink
Hi Olof,

It's good to see I'm not the only one who thinks
naming is important. :-)
Post by Olof Bjarnason
TestCase( object, object, ... )
TestCases( string )
Values( object, object, ... )
DataSource( string )
TestCase
TestCases
Values
ParameterSource
Yes... it made sense all alone but when
you line them up, it doesn't.
Post by Olof Bjarnason
I liked ValueSource( string ) better than ParameterSource
since it clings better to Values, so it is more analogous to
the TestCase, TestCases pair.
But then again the only thing differentiating TestCase and
TestCases is singular/plural form.
True
Post by Olof Bjarnason
I guess the hurdle is that the concept of a "named source" is
used in both pairs, but the name "Source" only in one. It is
1.
TestCase
TestCaseSource
Values
ValuesSource>
2.
TestCase
TestCases
Values
ValuesLookup (or something ..)
#1 is better in my view...

So we can extract some naming principles from this...

1. The inline and named source attributes for each type
should contain a common part like TestCase or Value that
ties them together.

2. The naming pattern of all inline sources together
should be the same. My preference is to use the "bare"
form, as we are doing so far.

3. The naming pattern of all named sources should
be the same, such as adding "Source" to the bare form.

4. Every other source type should work the same way
as we develop them... for example:
ExcelTestCases versus ExcelValues
DataBaseTestCaseSource versus DataBaseValueSource
[I'm not actually proposing those names, just illustrating.]

So, back to our immediate problem, I'm ready to go for
Post by Olof Bjarnason
TestCase
TestCaseSource
Values
ValueSource
I dropped the plural in Value source, simply because
a "source of values" is a "value source" in English.
I could live with it if others prefer, however.

What do you think?

Charlie
Post by Olof Bjarnason
Post by Charlie Poole
Charlie
-----Original Message-----
Behalf Of
Post by Charlie Poole
Olof Bjarnason
Sent: Tuesday, September 09, 2008 6:32 AM
Subject: Re: [nunit-developer] [nunitv3] RE: Another Question of
Naming
OK I understand the ValuesAttribute / DataSourceAttribute now.
And I agree with cliff that Value[s]Source is more readable than
Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[ValueSource("b-values")]int b) {
Assert.That( a+b == b+a );
}
Post by Charlie Poole
Those are possibilities...
Charlie
________________________________
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Subject: Re: [nunit-developer] Another Question of Naming
What about ValueSource or ValuesSource?
On Mon, Sep 8, 2008 at 6:57 PM, Charlie Poole
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for
Post by Charlie Poole
Post by Charlie Poole
parameterized tests, so I'm asking for ideas, opinions and
suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for
Post by Charlie Poole
Post by Charlie Poole
parameters, which NUnit then combines into test cases
using various
Post by Charlie Poole
Post by Charlie Poole
strategies.
If you go with full test cases, they can either be inline or
separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they can
be inline
Post by Charlie Poole
Post by Charlie Poole
or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
We'll be adding others - cases in a spreadsheet, etc. -
but these are
Post by Charlie Poole
Post by Charlie Poole
the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the same.
But DataSource strikes me as a bit too generic. It
doesn't tell me
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
that it's a set of single parm values rather than a
test case. It
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
sounds like it might have something to do with a DB. But
for the life
Post by Charlie Poole
Post by Charlie Poole
of me I can't think of anything better.
Ideas?
Charlie
---------------------------------------------------------------------
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Charlie Poole
Post by Charlie Poole
Moblin SDK & win great prizes Grand prize is a trip for
two to an
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks
cliff
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to
the Google
Post by Charlie Poole
Post by Charlie Poole
Groups "NUnitV3" group.
unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Olof Bjarnason
2008-09-12 21:52:48 UTC
Permalink
Post by Charlie Poole
Hi Olof,
It's good to see I'm not the only one who thinks
naming is important. :-)
Post by Olof Bjarnason
TestCase( object, object, ... )
TestCases( string )
Values( object, object, ... )
DataSource( string )
TestCase
TestCases
Values
ParameterSource
Yes... it made sense all alone but when
you line them up, it doesn't.
Post by Olof Bjarnason
I liked ValueSource( string ) better than ParameterSource
since it clings better to Values, so it is more analogous to
the TestCase, TestCases pair.
But then again the only thing differentiating TestCase and
TestCases is singular/plural form.
True
Post by Olof Bjarnason
I guess the hurdle is that the concept of a "named source" is
used in both pairs, but the name "Source" only in one. It is
1.
TestCase
TestCaseSource
Values
ValuesSource>
2.
TestCase
TestCases
Values
ValuesLookup (or something ..)
#1 is better in my view...
So we can extract some naming principles from this...
1. The inline and named source attributes for each type
should contain a common part like TestCase or Value that
ties them together.
2. The naming pattern of all inline sources together
should be the same. My preference is to use the "bare"
form, as we are doing so far.
3. The naming pattern of all named sources should
be the same, such as adding "Source" to the bare form.
4. Every other source type should work the same way
ExcelTestCases versus ExcelValues
DataBaseTestCaseSource versus DataBaseValueSource
[I'm not actually proposing those names, just illustrating.]
So, back to our immediate problem, I'm ready to go for
Post by Olof Bjarnason
TestCase
TestCaseSource
Values
ValueSource
I dropped the plural in Value source, simply because
a "source of values" is a "value source" in English.
I could live with it if others prefer, however.
What do you think?
I think my english is inferior to yours, so go for it!

:)

cya

/Olof
Post by Charlie Poole
Charlie
Post by Olof Bjarnason
Post by Charlie Poole
Charlie
-----Original Message-----
Behalf Of
Post by Charlie Poole
Olof Bjarnason
Sent: Tuesday, September 09, 2008 6:32 AM
Subject: Re: [nunit-developer] [nunitv3] RE: Another Question of
Naming
OK I understand the ValuesAttribute / DataSourceAttribute now.
And I agree with cliff that Value[s]Source is more readable than
Test]
public void AdditionIsCommutative(
[Values(0, 1, 42, 99)] int a,
[ValueSource("b-values")]int b) {
Assert.That( a+b == b+a );
}
Post by Charlie Poole
Those are possibilities...
Charlie
________________________________
Sent: Monday, September 08, 2008 9:24 PM
To: Charlie Poole
Subject: Re: [nunit-developer] Another Question of Naming
What about ValueSource or ValuesSource?
On Mon, Sep 8, 2008 at 6:57 PM, Charlie Poole
Post by Charlie Poole
Hi All,
I'm still not happy with one of the attribute names we are
using for
Post by Charlie Poole
Post by Charlie Poole
parameterized tests, so I'm asking for ideas, opinions and
suggestions.
Here's what we have now...
You can specify entire test cases together or individual
values for
Post by Charlie Poole
Post by Charlie Poole
parameters, which NUnit then combines into test cases
using various
Post by Charlie Poole
Post by Charlie Poole
strategies.
If you go with full test cases, they can either be inline or
separately named and referenced...
TestCase( 1.2, 3.0 ) // Inline
TestCases( "myCases") // Separate
Similarly, if you go with individual parameters, they can
be inline
Post by Charlie Poole
Post by Charlie Poole
or separate...
Values(12, 7, 3) // Inline
DataSource("myValues") // Separate
We'll be adding others - cases in a spreadsheet, etc. -
but these are
Post by Charlie Poole
Post by Charlie Poole
the four basic ones.
I'm quite happy with TestCase, TestCases and Values.
I find them quite readable and others have said the same.
But DataSource strikes me as a bit too generic. It
doesn't tell me
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
that it's a set of single parm values rather than a
test case. It
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
sounds like it might have something to do with a DB. But
for the life
Post by Charlie Poole
Post by Charlie Poole
of me I can't think of anything better.
Ideas?
Charlie
---------------------------------------------------------------------
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Charlie Poole
Post by Charlie Poole
Moblin SDK & win great prizes Grand prize is a trip for
two to an
Post by Charlie Poole
Post by Charlie Poole
Post by Charlie Poole
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--
thanks
cliff
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to
the Google
Post by Charlie Poole
Post by Charlie Poole
Groups "NUnitV3" group.
unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/nunitv3?hl=en
-~----------~----~----~----~------~----~------~--~---
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Hans Christian Falkenberg
2008-09-12 12:32:41 UTC
Permalink
Hi,

I was about to post a bug about this - but then I read the
documentation and realized this might be something up
for discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...

Issues:
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown]
2. The documentation specifies that it should
do the Wrong Thing (imho) :p

What I think should happen:
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc says "any")

What the documentation says (in both 2.4.8 and 2.5 alpha 3):
So long as any SetUp method runs without error, the TearDown method is
guaranteed to run. It will not run if a SetUp method fails or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will not run if a
TestFixtureSetUp method fails or throws an exception.

What actually happens (in both 2.4.8 and 2.5 alpha 3):
Running attached code:
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down

Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
Uncommenting FixtureSetUp exception:
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds


So [TearDown] doesn't behave as specified when a [SetUp] method throws an
exception, that's a bug report, right?

Except I think it *should* behave that way, so I was going to submit
a bug report on [TestFixtureTearDown] :)

Here's my reasoning:
Say you are acquiring two resources in the SetUp method:
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}

And of course the must be released:
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}

Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
acquiring r2 (and it can), we'll have to write this:

[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}

Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of TearDown
code duplication in SetUp. So here is how I decided to work around
this problem:

[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}

[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}

Not the worst of solution's, but now I'm basically doing NUnit's
work myself. And when the methods in TearDown *can* throw Exceptions
they will now be reported as if they were thrown in SetUp,
which is just to ask for someone being confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which behaves
like the NUnit doc says it should) and future NUnit versions
which might adhere to the documentation.

So... could we instead please change the NUnit behavior so that
other people don't have to write workarounds like this?

I know junit doesn't run TearDown (or didn't when I last used it a
year back) when SetUp throws, but junit didn't report test
exceptions when TearDown throwed either, so not much cause
to let them set the standard here...

A better argument against might be that people didn't write code
to check for null in their [TearDown] and [TestFixtureTearDown]
methods. But then their [TearDown] methods are already failing,
and they'll just have to fix their [TestFixtureTearDown] methods
when upgrading to 2.5, won't they?

+ Hans Christian



PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are files
(with Acquire==Create and Release==Delete) and it would hardly
be proper to use mocked files for unit testing when testing
writing to the OS - that would kind of void the entire testing.

PPS: Why should a unit test need files? Well - it does.
Charlie Poole
2008-09-12 18:34:46 UTC
Permalink
Hi Hans Christian,

We should discuss this behavior and try to get it right
(whatever we decide that is) for 2.5.

But before we get into it, because of the name of your
test, I'm wondering if you actually ran under NUnit.
When you run under resharper, no matter what version
of the NUnit framework your test references, you
get the semantics that resharper defines. That's
because this particular behavior is not implemented
in nunit.framework, but in nunit.core, which is
not used in this situation.

I'll run my own tests as well, but I thought I
should ask.


Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc says "any")
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5, won't they?
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Hans Christian Falkenberg
2008-09-12 20:12:15 UTC
Permalink
Err, well sorry about the name of the test - I originally used
it to submit a bug report to ReSharper because I saw different
behavior wrt. whether the [TearDown] method was invoked when
I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)

So the answer is most likely that I used both :)

After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and [TestFixtureTearDown]
so I read the documentation and then I wrote this mail.

+ Hans Christian

PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that
I started a discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we decide that is) for 2.5.
But before we get into it, because of the name of your
test, I'm wondering if you actually ran under NUnit.
When you run under resharper, no matter what version
of the NUnit framework your test references, you
get the semantics that resharper defines. That's
because this particular behavior is not implemented
in nunit.framework, but in nunit.core, which is
not used in this situation.
I'll run my own tests as well, but I thought I
should ask.
Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc says "any")
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5, won't they?
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-12 20:44:50 UTC
Permalink
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.

I want to do some research before I get back to you on the
"how it should be" part. I have a vague sense that we may
have changed the behavior from what the docs say as the
result of a user request and if so I'd like to get the
reasons for that request into the discussion.

One part I can explain: in the 2.4 docs I meant the phrase
"any setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it out for clarity.

In 2.5, there can be multiple SetUp methods, and the logic
of which TearDowns should and should not be run still
has to be determined. Since there is not likely to be
another 2.4 release, it's 2.5 that is the key to this
discussion.

Charlie

Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I originally
used it to submit a bug report to ReSharper because I saw
different behavior wrt. whether the [TearDown] method was
invoked when I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and
[TestFixtureTearDown] so I read the documentation and then I
wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that I
started a discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your test, I'm
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of the NUnit
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Hans Christian Falkenberg
2008-09-12 21:15:13 UTC
Permalink
Ah, I thought I had read somewhere that
,
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on the
"how it should be" part. I have a vague sense that we may
have changed the behavior from what the docs say as the
result of a user request and if so I'd like to get the
reasons for that request into the discussion.
One part I can explain: in the 2.4 docs I meant the phrase
"any setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it out for clarity.
In 2.5, there can be multiple SetUp methods, and the logic
of which TearDowns should and should not be run still
has to be determined. Since there is not likely to be
another 2.4 release, it's 2.5 that is the key to this
discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I originally
used it to submit a bug report to ReSharper because I saw
different behavior wrt. whether the [TearDown] method was
invoked when I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and
[TestFixtureTearDown] so I read the documentation and then I
wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that I
started a discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your test, I'm
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of the NUnit
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Hans Christian Falkenberg
2008-09-12 21:22:24 UTC
Permalink
(Let me just try again with batteries in the crazy keyboard)

Ah, I thought I had some place about support for multiple
[SetUp], but I couldn't remember if it was decided for or
against, so I read the 2.5 docs (which understandably aren't
up to date yet then) and concluded that there would only be
a single [SetUp].

With multiple ones, the conclusion is a given, I think.
Consider this:

[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}

[TearDown] public void TearDown1() {
Release1();
}

[TearDown] public void TearDown2() {
Release2();
}

Now, if SetUp1 succeeds and SetUp2 throws - what are you gonna do?
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and confusion.

To me it looks like the only intuitive and easily understandable
approach is:
1. If any [SetUp] method throws, no further [SetUp] methods will be
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people)
3. Correspondingly [TestFixture*], obviously

But feel free to correct me with coming up with something simpler :)

+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on the
"how it should be" part. I have a vague sense that we may
have changed the behavior from what the docs say as the
result of a user request and if so I'd like to get the
reasons for that request into the discussion.
One part I can explain: in the 2.4 docs I meant the phrase
"any setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it out for clarity.
In 2.5, there can be multiple SetUp methods, and the logic
of which TearDowns should and should not be run still
has to be determined. Since there is not likely to be
another 2.4 release, it's 2.5 that is the key to this
discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I originally
used it to submit a bug report to ReSharper because I saw
different behavior wrt. whether the [TearDown] method was
invoked when I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and
[TestFixtureTearDown] so I read the documentation and then I
wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that I
started a discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your test, I'm
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of the NUnit
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-12 23:54:05 UTC
Permalink
Hi Hans Christian,

I agree. Although we *could* partition SetUps into those
found in the top level, those found in the base, those
found in the base of the base, etc. I don't think I want
to. So either all or none of the TearDowns have to run
and none is pretty useless.

However, I think we can make a distinction between
SetUp, TestFixtureSetUp and higher level SetUp.

Charlie
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 2:22 PM
Subject: Re: [nunit-developer] Tearing down when SetUp fails?
(Let me just try again with batteries in the crazy keyboard)
Ah, I thought I had some place about support for multiple
[SetUp], but I couldn't remember if it was decided for or
against, so I read the 2.5 docs (which understandably aren't
up to date yet then) and concluded that there would only be a
single [SetUp].
With multiple ones, the conclusion is a given, I think.
[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}
[TearDown] public void TearDown1() {
Release1();
}
[TearDown] public void TearDown2() {
Release2();
}
Now, if SetUp1 succeeds and SetUp2 throws - what are you gonna do?
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and confusion.
To me it looks like the only intuitive and easily
1. If any [SetUp] method throws, no further [SetUp] methods will be
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people) 3. Correspondingly
[TestFixture*], obviously
But feel free to correct me with coming up with something simpler :)
+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction
Post by Charlie Poole
betweeen running resharper and nunit-console.
I want to do some research before I get back to you on the "how it
should be" part. I have a vague sense that we may have changed the
behavior from what the docs say as the result of a user
request and if
Post by Charlie Poole
so I'd like to get the reasons for that request into the discussion.
One part I can explain: in the 2.4 docs I meant the phrase
"any setup
Post by Charlie Poole
method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it out
Post by Charlie Poole
for clarity.
In 2.5, there can be multiple SetUp methods, and the logic of which
TearDowns should and should not be run still has to be determined.
Since there is not likely to be another 2.4 release, it's
2.5 that is
Post by Charlie Poole
the key to this discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I originally
used it to
Post by Charlie Poole
submit a bug report to ReSharper because I saw different behavior
wrt. whether the [TearDown] method was invoked when I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even NUnit does it
different between [TearDown] and [TestFixtureTearDown] so
I read the
Post by Charlie Poole
documentation and then I wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an
hour after my
Post by Charlie Poole
bug report that they would look on the issue) that I started a
discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your test, I'm
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of the NUnit
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
On Behalf Of
Post by Charlie Poole
Post by Charlie Poole
Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the TearDown
method is
guaranteed to run. It will not run if a SetUp method fails or
throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It
will not run
Post by Charlie Poole
Post by Charlie Poole
if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp] method
throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument is
null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad...
until you have
Post by Charlie Poole
Post by Charlie Poole
4-5 resources. Then there's suddenly a crazy amount of TearDown
code duplication in SetUp. So here is how I decided to
work around
Post by Charlie Poole
Post by Charlie Poole
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing NUnit's
work myself. And when the methods in TearDown *can* throw
Exceptions they will now be reported as if they were thrown in
SetUp, which is just to ask for someone being confused later.
But at least the code will release resources properly in
both NUnit
Post by Charlie Poole
Post by Charlie Poole
2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which behaves like
the NUnit doc says it should) and future NUnit versions
which might
Post by Charlie Poole
Post by Charlie Poole
adhere to the documentation.
So... could we instead please change the NUnit behavior so that
other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last
used it a
Post by Charlie Poole
Post by Charlie Poole
year back) when SetUp throws, but junit didn't report test
exceptions when TearDown throwed either, so not much
cause to let
Post by Charlie Poole
Post by Charlie Poole
them set the standard here...
A better argument against might be that people didn't
write code to
Post by Charlie Poole
Post by Charlie Poole
check for null in their [TearDown] and [TestFixtureTearDown]
methods. But then their [TearDown] methods are already
failing, and
Post by Charlie Poole
Post by Charlie Poole
they'll just have to fix their [TestFixtureTearDown]
methods when
Post by Charlie Poole
Post by Charlie Poole
upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are files
(with Acquire==Create and Release==Delete) and it would
hardly be
Post by Charlie Poole
Post by Charlie Poole
proper to use mocked files for unit testing when testing
writing to
Post by Charlie Poole
Post by Charlie Poole
the OS - that would kind of void the entire testing.
PPS: Why should a unit test need files? Well - it does.
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Kelly Anderson
2008-09-15 15:37:01 UTC
Permalink
Multiple Setups seems like a bad idea to me. If you need multiple
setups, why not have multiple TestFixture classes?

What order would they run in? How do you associate a particular SetUp
with a particular TearDown... this sounds like opening up a can of
worms with no real payback.

Is this really being considered seriously?

-Kelly

On Fri, Sep 12, 2008 at 3:22 PM, Hans Christian Falkenberg
Post by Hans Christian Falkenberg
(Let me just try again with batteries in the crazy keyboard)
Ah, I thought I had some place about support for multiple
[SetUp], but I couldn't remember if it was decided for or
against, so I read the 2.5 docs (which understandably aren't
up to date yet then) and concluded that there would only be
a single [SetUp].
With multiple ones, the conclusion is a given, I think.
[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}
[TearDown] public void TearDown1() {
Release1();
}
[TearDown] public void TearDown2() {
Release2();
}
Now, if SetUp1 succeeds and SetUp2 throws - what are you gonna do?
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and confusion.
To me it looks like the only intuitive and easily understandable
1. If any [SetUp] method throws, no further [SetUp] methods will be
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people)
3. Correspondingly [TestFixture*], obviously
But feel free to correct me with coming up with something simpler :)
+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on the
"how it should be" part. I have a vague sense that we may
have changed the behavior from what the docs say as the
result of a user request and if so I'd like to get the
reasons for that request into the discussion.
One part I can explain: in the 2.4 docs I meant the phrase
"any setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it out for clarity.
In 2.5, there can be multiple SetUp methods, and the logic
of which TearDowns should and should not be run still
has to be determined. Since there is not likely to be
another 2.4 release, it's 2.5 that is the key to this
discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I originally
used it to submit a bug report to ReSharper because I saw
different behavior wrt. whether the [TearDown] method was
invoked when I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and
[TestFixtureTearDown] so I read the documentation and then I
wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that I
started a discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your test, I'm
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of the NUnit
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Olof Bjarnason
2008-09-16 19:45:00 UTC
Permalink
Start simple, extent if needed ...

So one Setup, one TearDown is my vote
Post by Kelly Anderson
Multiple Setups seems like a bad idea to me. If you need multiple
setups, why not have multiple TestFixture classes?
What order would they run in? How do you associate a particular SetUp
with a particular TearDown... this sounds like opening up a can of
worms with no real payback.
Is this really being considered seriously?
-Kelly
On Fri, Sep 12, 2008 at 3:22 PM, Hans Christian Falkenberg
Post by Hans Christian Falkenberg
(Let me just try again with batteries in the crazy keyboard)
Ah, I thought I had some place about support for multiple
[SetUp], but I couldn't remember if it was decided for or
against, so I read the 2.5 docs (which understandably aren't
up to date yet then) and concluded that there would only be
a single [SetUp].
With multiple ones, the conclusion is a given, I think.
[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}
[TearDown] public void TearDown1() {
Release1();
}
[TearDown] public void TearDown2() {
Release2();
}
Now, if SetUp1 succeeds and SetUp2 throws - what are you gonna do?
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and confusion.
To me it looks like the only intuitive and easily understandable
1. If any [SetUp] method throws, no further [SetUp] methods will be
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people)
3. Correspondingly [TestFixture*], obviously
But feel free to correct me with coming up with something simpler :)
+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on the
"how it should be" part. I have a vague sense that we may
have changed the behavior from what the docs say as the
result of a user request and if so I'd like to get the
reasons for that request into the discussion.
One part I can explain: in the 2.4 docs I meant the phrase
"any setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it out for clarity.
In 2.5, there can be multiple SetUp methods, and the logic
of which TearDowns should and should not be run still
has to be determined. Since there is not likely to be
another 2.4 release, it's 2.5 that is the key to this
discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I originally
used it to submit a bug report to ReSharper because I saw
different behavior wrt. whether the [TearDown] method was
invoked when I used
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even NUnit
does it different between [TearDown] and
[TestFixtureTearDown] so I read the documentation and then I
wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an hour
after my bug report that they would look on the issue) that I
started a discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your test, I'm
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of the NUnit
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
Behalf Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown method is
guaranteed to run. It will not run if a SetUp method fails
or throws an
exception.
and
So long as any TestFixtureSetUp method runs without error, the
TestFixtureTearDown method is guaranteed to run. It will
not run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a [SetUp]
method throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its argument
is null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I decided
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically doing
NUnit's work myself. And when the methods in TearDown *can*
throw Exceptions they will now be reported as if they were
thrown in SetUp, which is just to ask for someone being
confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3 (which
behaves like the NUnit doc says it should) and future NUnit
versions which might adhere to the documentation.
So... could we instead please change the NUnit behavior so
that other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I last used
it a year back) when SetUp throws, but junit didn't report
test exceptions when TearDown throwed either, so not much
cause to let them set the standard here...
A better argument against might be that people didn't write
code to check for null in their [TearDown] and
[TestFixtureTearDown] methods. But then their [TearDown]
methods are already failing, and they'll just have to fix
their [TestFixtureTearDown] methods when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are
files (with Acquire==Create and Release==Delete) and it would
hardly be proper to use mocked files for unit testing when
testing writing to the OS - that would kind of void the
entire testing.
PPS: Why should a unit test need files? Well - it does.
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-17 03:46:58 UTC
Permalink
Hi Guys,

This isn't planned - it's implemented. :-)

The motivation for multiple setups is to allow use of
inheritance, with base class setups called before
derived class and teardowns called in the reverse
order. [Note I'm using lower case to mean "all kinds
of setup.]

Having multiple setups in the same class - not just
in the base is a fallout of the implementation. In
other words, the simplest thing to do was to just
let it happen - it would take more code ot detect
and prevent it. We can do that, but I always like
to have a reason before doing extra work. :-)h

It's possible that figuring out how to handle
failures may serve as the reason for that work,
but I don't see it yet.

If we change to always running teardown if setup
was run, then there is no problem. NUnit would
just treat multiple setups at the same level as if
it were one big setup. It's only if we have to
discriminate among them that it gets hard. In that
case, I'd write the code to prevent it, but otherwise,
I'd just let it happen.

Charlie
-----Original Message-----
Behalf Of Olof Bjarnason
Sent: Tuesday, September 16, 2008 12:45 PM
Subject: Re: [nunit-developer] Tearing down when SetUp fails?
Start simple, extent if needed ...
So one Setup, one TearDown is my vote
Post by Kelly Anderson
Multiple Setups seems like a bad idea to me. If you need multiple
setups, why not have multiple TestFixture classes?
What order would they run in? How do you associate a
particular SetUp
Post by Kelly Anderson
with a particular TearDown... this sounds like opening up a can of
worms with no real payback.
Is this really being considered seriously?
-Kelly
On Fri, Sep 12, 2008 at 3:22 PM, Hans Christian Falkenberg
Post by Hans Christian Falkenberg
(Let me just try again with batteries in the crazy keyboard)
Ah, I thought I had some place about support for multiple [SetUp],
but I couldn't remember if it was decided for or against,
so I read
Post by Kelly Anderson
Post by Hans Christian Falkenberg
the 2.5 docs (which understandably aren't up to date yet then) and
concluded that there would only be a single [SetUp].
With multiple ones, the conclusion is a given, I think.
[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}
[TearDown] public void TearDown1() {
Release1();
}
[TearDown] public void TearDown2() {
Release2();
}
Now, if SetUp1 succeeds and SetUp2 throws - what are you gonna do?
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and
confusion.
Post by Kelly Anderson
Post by Hans Christian Falkenberg
To me it looks like the only intuitive and easily understandable
1. If any [SetUp] method throws, no further [SetUp] methods will be
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people) 3. Correspondingly
[TestFixture*], obviously
But feel free to correct me with coming up with something
simpler :)
Post by Kelly Anderson
Post by Hans Christian Falkenberg
+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on
the "how it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
should be" part. I have a vague sense that we may have
changed the
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
behavior from what the docs say as the result of a user
request and
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
if so I'd like to get the reasons for that request into the
discussion.
One part I can explain: in the 2.4 docs I meant the phrase "any
setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have spelled it
out for clarity.
In 2.5, there can be multiple SetUp methods, and the
logic of which
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
TearDowns should and should not be run still has to be
determined.
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Since there is not likely to be another 2.4 release, it's
2.5 that
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
is the key to this discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I
originally used it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
to submit a bug report to ReSharper because I saw different
behavior wrt. whether the [TearDown] method was invoked
when I used
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even
NUnit does it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
different between [TearDown] and [TestFixtureTearDown] so I read
the documentation and then I wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an
hour after
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
my bug report that they would look on the issue) that I
started a
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your
test, I'm
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of
the NUnit
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
On Behalf
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
method is
guaranteed to run. It will not run if a SetUp
method fails or
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
throws an
exception.
and
So long as any TestFixtureSetUp method runs without
error, the
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
TestFixtureTearDown method is guaranteed to run. It
will not
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a
[SetUp] method
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its
argument is
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I
decided to
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically
doing NUnit's
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
work myself. And when the methods in TearDown *can* throw
Exceptions they will now be reported as if they were thrown in
SetUp, which is just to ask for someone being confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3
(which behaves
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
like the NUnit doc says it should) and future NUnit versions
which might adhere to the documentation.
So... could we instead please change the NUnit
behavior so that
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I
last used it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
a year back) when SetUp throws, but junit didn't report test
exceptions when TearDown throwed either, so not much
cause to let
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
them set the standard here...
A better argument against might be that people didn't
write code
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
to check for null in their [TearDown] and
[TestFixtureTearDown]
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
methods. But then their [TearDown] methods are already
failing,
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
and they'll just have to fix their
[TestFixtureTearDown] methods
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are files
(with Acquire==Create and Release==Delete) and it
would hardly be
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
proper to use mocked files for unit testing when
testing writing
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
to the OS - that would kind of void the entire testing.
PPS: Why should a unit test need files? Well - it does.
---------------------------------------------------------------------
Post by Kelly Anderson
Post by Hans Christian Falkenberg
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Moblin SDK & win great prizes Grand prize is a trip for two to an
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
----------------------------------------------------------------------
Post by Kelly Anderson
--- This SF.Net email is sponsored by the Moblin Your Move
Developer's
Post by Kelly Anderson
challenge Build the coolest Linux based applications with
Moblin SDK &
Post by Kelly Anderson
win great prizes Grand prize is a trip for two to an Open
Source event
Post by Kelly Anderson
anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Kelly Anderson
2008-09-17 20:47:49 UTC
Permalink
Ok Charlie, I totally understand the SetUp TearDown thing in the
inheritance model. That makes sense, because it's just like
constructors and destructors (at least in C++, let's not get into the
weird end of life of C# objects) and the order they are executed makes
sense because of that prior experience. I think I've done this myself,
in fact. It was useful.

That being said, if you have two SetUps in the same class, I would
assume they are all executed prior to calling each test. But how do
you know which one is executed first? Or must the order of execution
not matter to be safe?

-Kelly

On Tue, Sep 16, 2008 at 9:46 PM, Charlie Poole
Post by Charlie Poole
Hi Guys,
This isn't planned - it's implemented. :-)
The motivation for multiple setups is to allow use of
inheritance, with base class setups called before
derived class and teardowns called in the reverse
order. [Note I'm using lower case to mean "all kinds
of setup.]
Having multiple setups in the same class - not just
in the base is a fallout of the implementation. In
other words, the simplest thing to do was to just
let it happen - it would take more code ot detect
and prevent it. We can do that, but I always like
to have a reason before doing extra work. :-)h
It's possible that figuring out how to handle
failures may serve as the reason for that work,
but I don't see it yet.
If we change to always running teardown if setup
was run, then there is no problem. NUnit would
just treat multiple setups at the same level as if
it were one big setup. It's only if we have to
discriminate among them that it gets hard. In that
case, I'd write the code to prevent it, but otherwise,
I'd just let it happen.
Charlie
-----Original Message-----
Behalf Of Olof Bjarnason
Sent: Tuesday, September 16, 2008 12:45 PM
Subject: Re: [nunit-developer] Tearing down when SetUp fails?
Start simple, extent if needed ...
So one Setup, one TearDown is my vote
Post by Kelly Anderson
Multiple Setups seems like a bad idea to me. If you need multiple
setups, why not have multiple TestFixture classes?
What order would they run in? How do you associate a
particular SetUp
Post by Kelly Anderson
with a particular TearDown... this sounds like opening up a can of
worms with no real payback.
Is this really being considered seriously?
-Kelly
On Fri, Sep 12, 2008 at 3:22 PM, Hans Christian Falkenberg
Post by Hans Christian Falkenberg
(Let me just try again with batteries in the crazy keyboard)
Ah, I thought I had some place about support for multiple [SetUp],
but I couldn't remember if it was decided for or against,
so I read
Post by Kelly Anderson
Post by Hans Christian Falkenberg
the 2.5 docs (which understandably aren't up to date yet then) and
concluded that there would only be a single [SetUp].
With multiple ones, the conclusion is a given, I think.
[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}
[TearDown] public void TearDown1() {
Release1();
}
[TearDown] public void TearDown2() {
Release2();
}
Now, if SetUp1 succeeds and SetUp2 throws - what are you gonna do?
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and
confusion.
Post by Kelly Anderson
Post by Hans Christian Falkenberg
To me it looks like the only intuitive and easily understandable
1. If any [SetUp] method throws, no further [SetUp] methods will be
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people) 3. Correspondingly
[TestFixture*], obviously
But feel free to correct me with coming up with something
simpler :)
Post by Kelly Anderson
Post by Hans Christian Falkenberg
+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on
the "how it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
should be" part. I have a vague sense that we may have
changed the
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
behavior from what the docs say as the result of a user
request and
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
if so I'd like to get the reasons for that request into the
discussion.
One part I can explain: in the 2.4 docs I meant the phrase "any
setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have spelled it
out for clarity.
In 2.5, there can be multiple SetUp methods, and the
logic of which
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
TearDowns should and should not be run still has to be
determined.
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Since there is not likely to be another 2.4 release, it's
2.5 that
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
is the key to this discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I
originally used it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
to submit a bug report to ReSharper because I saw different
behavior wrt. whether the [TearDown] method was invoked
when I used
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even
NUnit does it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
different between [TearDown] and [TestFixtureTearDown] so I read
the documentation and then I wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an
hour after
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
my bug report that they would look on the issue) that I
started a
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your
test, I'm
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of
the NUnit
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not used in this
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
On Behalf
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The documentation
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any [TestFixtureSetUp] method,
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
method is
guaranteed to run. It will not run if a SetUp
method fails or
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
throws an
exception.
and
So long as any TestFixtureSetUp method runs without
error, the
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
TestFixtureTearDown method is guaranteed to run. It
will not
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
Tests run: 1, Failures: 1, Not run: 0, Time: 0.030 seconds
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
Tests run: 1, Failures: 1, Not run: 0, Time: 0.028 seconds
So [TearDown] doesn't behave as specified when a
[SetUp] method
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I was going to
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its
argument is
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can occur while
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad... until you
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I
decided to
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically
doing NUnit's
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
work myself. And when the methods in TearDown *can* throw
Exceptions they will now be reported as if they were thrown in
SetUp, which is just to ask for someone being confused later.
But at least the code will release resources properly in both
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3
(which behaves
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
like the NUnit doc says it should) and future NUnit versions
which might adhere to the documentation.
So... could we instead please change the NUnit
behavior so that
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I
last used it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
a year back) when SetUp throws, but junit didn't report test
exceptions when TearDown throwed either, so not much
cause to let
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
them set the standard here...
A better argument against might be that people didn't
write code
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
to check for null in their [TearDown] and
[TestFixtureTearDown]
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
methods. But then their [TearDown] methods are already
failing,
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
and they'll just have to fix their
[TestFixtureTearDown] methods
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt. acquiring 4-5
resources before a unit tests... Well, the resources are files
(with Acquire==Create and Release==Delete) and it
would hardly be
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
proper to use mocked files for unit testing when
testing writing
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
to the OS - that would kind of void the entire testing.
PPS: Why should a unit test need files? Well - it does.
---------------------------------------------------------------------
Post by Kelly Anderson
Post by Hans Christian Falkenberg
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Moblin SDK & win great prizes Grand prize is a trip for two to an
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
----------------------------------------------------------------------
Post by Kelly Anderson
--- This SF.Net email is sponsored by the Moblin Your Move
Developer's
Post by Kelly Anderson
challenge Build the coolest Linux based applications with
Moblin SDK &
Post by Kelly Anderson
win great prizes Grand prize is a trip for two to an Open
Source event
Post by Kelly Anderson
anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-09-17 21:29:15 UTC
Permalink
Hi Kelly,
Post by Kelly Anderson
Ok Charlie, I totally understand the SetUp TearDown thing in
the inheritance model. That makes sense, because it's just
like constructors and destructors (at least in C++, let's not
get into the weird end of life of C# objects) and the order
they are executed makes sense because of that prior
experience. I think I've done this myself, in fact. It was useful.
I undersand what you mean. And it's a fairly natural way to think
as well - which is why folks asked for it.
Post by Kelly Anderson
That being said, if you have two SetUps in the same class, I
would assume they are all executed prior to calling each
test. But how do you know which one is executed first? Or
must the order of execution not matter to be safe?
My answer is that it must not matter. This will, of course,
need to be documented.

Charlie
Post by Kelly Anderson
-Kelly
On Tue, Sep 16, 2008 at 9:46 PM, Charlie Poole
Post by Charlie Poole
Hi Guys,
This isn't planned - it's implemented. :-)
The motivation for multiple setups is to allow use of inheritance,
with base class setups called before derived class and teardowns
called in the reverse order. [Note I'm using lower case to
mean "all
Post by Charlie Poole
kinds of setup.]
Having multiple setups in the same class - not just in the
base is a
Post by Charlie Poole
fallout of the implementation. In other words, the simplest
thing to
Post by Charlie Poole
do was to just let it happen - it would take more code ot
detect and
Post by Charlie Poole
prevent it. We can do that, but I always like to have a
reason before
Post by Charlie Poole
doing extra work. :-)h
It's possible that figuring out how to handle failures may
serve as
Post by Charlie Poole
the reason for that work, but I don't see it yet.
If we change to always running teardown if setup was run,
then there
Post by Charlie Poole
is no problem. NUnit would just treat multiple setups at the same
level as if it were one big setup. It's only if we have to
discriminate among them that it gets hard. In that case,
I'd write the
Post by Charlie Poole
code to prevent it, but otherwise, I'd just let it happen.
Charlie
-----Original Message-----
Behalf Of
Post by Charlie Poole
Olof Bjarnason
Sent: Tuesday, September 16, 2008 12:45 PM
Subject: Re: [nunit-developer] Tearing down when SetUp fails?
Start simple, extent if needed ...
So one Setup, one TearDown is my vote
Post by Kelly Anderson
Multiple Setups seems like a bad idea to me. If you need
multiple
Post by Charlie Poole
Post by Kelly Anderson
setups, why not have multiple TestFixture classes?
What order would they run in? How do you associate a
particular SetUp
Post by Kelly Anderson
with a particular TearDown... this sounds like opening
up a can of
Post by Charlie Poole
Post by Kelly Anderson
worms with no real payback.
Is this really being considered seriously?
-Kelly
On Fri, Sep 12, 2008 at 3:22 PM, Hans Christian Falkenberg
Post by Hans Christian Falkenberg
(Let me just try again with batteries in the crazy keyboard)
Ah, I thought I had some place about support for
multiple [SetUp],
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
but I couldn't remember if it was decided for or against,
so I read
Post by Kelly Anderson
Post by Hans Christian Falkenberg
the 2.5 docs (which understandably aren't up to date
yet then) and
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
concluded that there would only be a single [SetUp].
With multiple ones, the conclusion is a given, I think.
[SetUp] public void SetUp1() {
Acquire1();
}
[SetUp] public void SetUp2() {
Acquire2();
}
[TearDown] public void TearDown1() {
Release1();
}
[TearDown] public void TearDown2() {
Release2();
}
Now, if SetUp1 succeeds and SetUp2 throws - what are
you gonna do?
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Annotate which [TearDown] belongs to which [SetUp]?
That's the sure road to overhead, strange constraints and
confusion.
Post by Kelly Anderson
Post by Hans Christian Falkenberg
To me it looks like the only intuitive and easily
understandable
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
1. If any [SetUp] method throws, no further [SetUp]
methods will be
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
called.
2. All [TearDown] methods will *always* be called.
(Just check for null values, people) 3. Correspondingly
[TestFixture*], obviously
But feel free to correct me with coming up with something
simpler :)
Post by Kelly Anderson
Post by Hans Christian Falkenberg
+ Hans Christian
Post by Charlie Poole
OK, I re-read your first mail and I now see you did make a
distinction betweeen running resharper and nunit-console.
I want to do some research before I get back to you on
the "how it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
should be" part. I have a vague sense that we may have
changed the
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
behavior from what the docs say as the result of a user
request and
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
if so I'd like to get the reasons for that request into the
discussion.
One part I can explain: in the 2.4 docs I meant the
phrase "any
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
setup method" to mean "either SetUp or TestFixtureSetUp."
It's still not working as described, but I should have
spelled it
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
out for clarity.
In 2.5, there can be multiple SetUp methods, and the
logic of which
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
TearDowns should and should not be run still has to be
determined.
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Since there is not likely to be another 2.4 release, it's
2.5 that
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
is the key to this discussion.
Charlie
Charlie
-----Original Message-----
Sent: Friday, September 12, 2008 1:12 PM
To: Charlie Poole
Subject: RE: [nunit-developer] Tearing down when SetUp fails?
Err, well sorry about the name of the test - I
originally used it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
to submit a bug report to ReSharper because I saw different
behavior wrt. whether the [TearDown] method was invoked
when I used
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
1) nunit-console.exe (see my exact command in orig mail) and
2) Right click -> Run Unit Tests (with ReSharper in VS)
So the answer is most likely that I used both :)
After submitting that bug report I noticed that even
NUnit does it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
different between [TearDown] and
[TestFixtureTearDown] so I read
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
the documentation and then I wrote this mail.
+ Hans Christian
PS! I also told the ReSharper guys (who replied just an
hour after
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
my bug report that they would look on the issue) that I
started a
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
discussion on what the behavior should be in NUnit.
Post by Charlie Poole
Hi Hans Christian,
We should discuss this behavior and try to get it right
(whatever we
Post by Charlie Poole
decide that is) for 2.5.
But before we get into it, because of the name of your
test, I'm
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
wondering if you actually ran under NUnit.
When you run under resharper, no matter what version of
the NUnit
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
framework your test references, you get the semantics that
resharper
Post by Charlie Poole
defines. That's because this particular behavior is not
implemented in
Post by Charlie Poole
nunit.framework, but in nunit.core, which is not
used in this
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
situation.
I'll run my own tests as well, but I thought I should ask.
Charlie.
-----Original Message-----
On Behalf
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
Of Hans Christian Falkenberg
Sent: Friday, September 12, 2008 5:33 AM
Subject: [nunit-developer] Tearing down when SetUp fails?
Hi,
I was about to post a bug about this - but then I read the
documentation and realized this might be something up for
discussion. Please let me know if it's there's a previous
conclusion on this and I'll just submit the bug report...
1. TearDown behavior is inconsistent between
[TearDown] and [TestFixtureTearDown] 2. The
documentation
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
specifies that it should
do the Wrong Thing (imho) :p
If there is an exception in any [SetUp] method, still run
all [TearDown] methods.
If there is an exception in any
[TestFixtureSetUp] method,
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
still run all [TestFixtureTearDown] methods.
(I realize only one of each method is allowed, but the doc
says "any")
Post by Charlie Poole
So long as any SetUp method runs without error, the
TearDown
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
method is
guaranteed to run. It will not run if a SetUp
method fails or
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
throws an
exception.
and
So long as any TestFixtureSetUp method runs without
error, the
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
TestFixtureTearDown method is guaranteed to run. It
will not
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
run if a
TestFixtureSetUp method fails or throws an exception.
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.Setting up
Tearing down
FFixture Tearing down
0.030 seconds
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
c:\>"c:\Program Files\NUnit 2.4.8\bin\nunit-console.exe"
/nologo bin\Debug\Dummy.dll
Fixture Setting up
.F
0.028 seconds
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
So [TearDown] doesn't behave as specified when a
[SetUp] method
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
throws an exception, that's a bug report, right?
Except I think it *should* behave that way, so I
was going to
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
submit a bug report on [TestFixtureTearDown] :)
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Now, since Resource.Release will just return if its
argument is
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
null and never throws, everything is fine. Except if the
documentation is to believed: If an exception can
occur while
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
[SetUp] public void SetUp() {
r1 = Resources.Acquire1();
try {
r2 = Resources.Acquire2();
} catch {
Resources.Release(r1);
throw;
}
}
Which I think is bad, but I guess not *that* bad...
until you
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
have 4-5 resources. Then there's suddenly a crazy amount of
TearDown code duplication in SetUp. So here is how I
decided to
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
[SetUp] public void SetUp() {
setUpCalled = true;
r1 = Resources.Acquire1();
r2 = Resources.Acquire2();
}
[TearDown] public void TearDown() {
if (!setUpCalled) return;
setUpCalled = false;
Resources.Release(r1);
r1 = null;
Resources.Release(r2);
r2 = null;
}
Not the worst of solution's, but now I'm basically
doing NUnit's
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
work myself. And when the methods in TearDown *can* throw
Exceptions they will now be reported as if they
were thrown in
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
SetUp, which is just to ask for someone being
confused later.
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
But at least the code will release resources
properly in both
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
NUnit 2.4.8 (and 2.5 alpha 3), ReSharper 4.1.933.3
(which behaves
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
like the NUnit doc says it should) and future NUnit
versions
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
which might adhere to the documentation.
So... could we instead please change the NUnit
behavior so that
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
other people don't have to write workarounds like this?
I know junit doesn't run TearDown (or didn't when I
last used it
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
a year back) when SetUp throws, but junit didn't
report test
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
exceptions when TearDown throwed either, so not much
cause to let
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
them set the standard here...
A better argument against might be that people didn't
write code
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
to check for null in their [TearDown] and
[TestFixtureTearDown]
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
methods. But then their [TearDown] methods are already
failing,
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
and they'll just have to fix their
[TestFixtureTearDown] methods
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
when upgrading to 2.5,
won't they?
Post by Charlie Poole
+ Hans Christian
PS: If you are wondering wth. I'm thinking wrt.
acquiring 4-5
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
resources before a unit tests... Well, the
resources are files
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
(with Acquire==Create and Release==Delete) and it
would hardly be
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
proper to use mocked files for unit testing when
testing writing
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Post by Charlie Poole
Post by Charlie Poole
to the OS - that would kind of void the entire testing.
PPS: Why should a unit test need files? Well - it does.
---------------------------------------------------------------------
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
---- This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Moblin SDK & win great prizes Grand prize is a trip for
two to an
Post by Charlie Poole
Post by Kelly Anderson
Post by Hans Christian Falkenberg
Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
---------------------------------------------------------------------
Post by Charlie Poole
-
Post by Kelly Anderson
--- This SF.Net email is sponsored by the Moblin Your Move
Developer's
Post by Kelly Anderson
challenge Build the coolest Linux based applications with
Moblin SDK &
Post by Kelly Anderson
win great prizes Grand prize is a trip for two to an Open
Source event
Post by Kelly Anderson
anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge Build the coolest Linux based applications with
Moblin SDK
Post by Charlie Poole
& win great prizes Grand prize is a trip for two to an Open Source
event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
----------------------------------------------------------------------
Post by Charlie Poole
--- This SF.Net email is sponsored by the Moblin Your Move
Developer's
Post by Charlie Poole
challenge Build the coolest Linux based applications with
Moblin SDK &
Post by Charlie Poole
win great prizes Grand prize is a trip for two to an Open
Source event
Post by Charlie Poole
anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
--------------------------------------------------------------
-----------
This SF.Net email is sponsored by the Moblin Your Move
Developer's challenge Build the coolest Linux based
applications with Moblin SDK & win great prizes Grand prize
is a trip for two to an Open Source event anywhere in the
world http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
nunit-developer mailing list
https://lists.sourceforge.net/lists/listinfo/nunit-developer
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Kelly Anderson
2008-09-18 16:05:37 UTC
Permalink
On Wed, Sep 17, 2008 at 3:29 PM, Charlie Poole
Post by Charlie Poole
Hi Kelly,
Post by Kelly Anderson
Ok Charlie, I totally understand the SetUp TearDown thing in
the inheritance model. That makes sense, because it's just
like constructors and destructors (at least in C++, let's not
get into the weird end of life of C# objects) and the order
they are executed makes sense because of that prior
experience. I think I've done this myself, in fact. It was useful.
I undersand what you mean. And it's a fairly natural way to think
as well - which is why folks asked for it.
Agreed.
Post by Charlie Poole
Post by Kelly Anderson
That being said, if you have two SetUps in the same class, I
would assume they are all executed prior to calling each
test. But how do you know which one is executed first? Or
must the order of execution not matter to be safe?
My answer is that it must not matter. This will, of course,
need to be documented.
Would be a good thing to stick in the NUnit Wiki... if there is one.

-Kelly

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
Charlie Poole
2008-10-03 19:17:42 UTC
Permalink
Hi Kelly,
Post by Kelly Anderson
Post by Charlie Poole
Post by Kelly Anderson
That being said, if you have two SetUps in the same class, I would
assume they are all executed prior to calling each test.
But how do
Post by Charlie Poole
Post by Kelly Anderson
you know which one is executed first? Or must the order of
execution
Post by Charlie Poole
Post by Kelly Anderson
not matter to be safe?
My answer is that it must not matter. This will, of course,
need to be
Post by Charlie Poole
documented.
Would be a good thing to stick in the NUnit Wiki... if there is one.
Agreed.

Charlie



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

Loading...