14. Testing¶

14.1. Assertions¶

A key notion of testing is the ability to make a logical assertion about something that generally must hold true if the test is to pass.

Assertions are not a standard language feature in C#. Instead, there are a number of classes that provide functions for assertion handling. In the framework we are using for unit testing (NUnit), a class named Assert supports assertion testing.

In our tests, we make use of an assertion method, Assert.IsTrue() to determine whether an assertion is successful. If the variable or expression passed to this method is false, the assertion fails.

Here are some examples of assertions:

• Assert.IsTrue(true): The assertion is trivially successful, because the boolean value true is true.
• Assert.IsTrue(false): The assertion is not successful, because the boolean value false is not true!
• Assert.IsFalse(false): This assertion is successful, because false is, of course, false.
• Assert.IsTrue(5 > 0): Success
• Assert.IsTrue(0 > 5): Failure

There are many available assertion methods. In our tests, we use Assert.IsTrue(), which works for everything we want to test. Other assertion methods do their magic rather similarly, because every assertion method ultimately must determine whether what is being tested is true or false.

14.2. Attributes¶

Besides assertions, a building block of testing (in C# and beyond) comes in the form of attributes. Attributes are an additional piece of information that can be attached to classes, variables, and methods in C#. There are two attributes of interest to us:

• [TestFixture]: This indicates that a class is being used for testing purposes.
• [Test]: This indicates that a method is one of the methods in a class being used for testing purposes.

Without these annotations, classes and methods will not be used for testing purposes. This allows a class to have some methods that are used for testing while other methods are ignored.

In the remainder of this section, we’re going to take a look at the strategy for testing the Rational class. In general, your goal is to ensure that the entire class is tested. It is easier said than done. In later courses (Software Engineering) you would learn about strategies for coverage testing.

Our strategy will be as follows:

• Test the constructor and make sure the representation of the rational number is sound. If the constructor isn’t initializing an instance properly, it is likely that little else in the class will work properly.
• Then test the rest of the class. Whenever possible, group the tests in some logical way. In the case of the Rational class, there are three general categories (and one rather special one): arithmetic operations, comparisons, and conversions. In addition, there is the parsing test, which ensures that we can convert strings representing fractions into properly initialized (and reduced) rational numbers.

Let’s get started.

14.3. Testing the Constructor¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

      [Test()]
      public void ConstructorTest()
      {
         Rational r = new Rational(3, 5);
         Assert.IsTrue(r.GetNumerator() == 3);
         Assert.IsTrue(r.GetDenominator() == 5);
         r = new Rational(3, -5);
         Assert.IsTrue(r.GetNumerator() == -3);
         Assert.IsTrue(r.GetDenominator() == 5);
         r = new Rational(6, 10);
         Assert.IsTrue(r.GetNumerator() == 3);
         Assert.IsTrue(r.GetDenominator() == 5);
         r = new Rational(125, 1);
         Assert.IsTrue(r.GetNumerator() == 125);
         Assert.IsTrue(r.GetDenominator() == 1);
      }

Testing the constructor is fairly straightforward. We essentially test three basic cases:

• Test whether a basic rational number can be constructed. In the above, we test for 3/5, 3/-5, 6/10, and 125. Per the implementation of the Rational class (how we defined it), these should result in fractions with numerators of 3, -3, 3, and 12; and denominators of 5, 5, 5, and 1, respectively.

• As you can observe from the code, we perform basic assertion testing to ensure that the numerators and denominators are what we expect. For example:

  Assert.IsTrue(r.GetNumerator() == 3)
  

  Tests whether the newly minted rational number, Rational(3, 5), actually has the expected numerator of 3.

• If we are able to get through the entire code of the ConstructorTest() method, our constructor test is a success. Otherwise, it is a failure.

We’ll look at how to actually run our tests in a bit but let’s continue taking a look at how the rest of our testing is done.

14.4. Testing Rational Comparisons¶

1
2
3
4
5
6
7
8
9

      [Test()]
      public void BasicComparisonTests() {
         Rational r1 = new Rational(-3, 6);
         Rational r2 = new Rational(2, 4);
         Rational r3 = new Rational(1, 2);
         Assert.IsTrue(r1.CompareTo(r2) < 0);
         Assert.IsTrue(r2.CompareTo(r1) > 0);
         Assert.IsTrue(r2.CompareTo(r3) == 0);
      }

It is pretty well established by now that the ability to compare is of fundamental importance whenever we are talking about data. Everything we do, especially when it comes to searching (finding a value) and sorting (putting values in order) depends on comparison.

In this test, we construct a few Rational instances (r1, r2, and r3) and perform at least one test for each of the essential operators (>, <, and =). Recall from our earlier discussion of the Rational class that the CompareTo method return a value < 0 when one Rational is less than another. It returns a number > 0 for greater than, and == 0 for equal to.

If any one of these comparisons fails, this means that we cannot rely on the ability to compare Rational numbers. This will likely prevent other tests from working, such as the arithmetic tests, which rely on the ability to test whether a computed result matches an expected result (e.g. 1/4 + 2/4 == 3/4).

14.5. Testing Rational Arithmetic¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

      [Test()]
      public void BasicArithmeticTests() {
         Rational r, r1, r2;
         r1 = new Rational(47, 64);
         r2 = new Rational(-11, 64);

         r = r1.Add(r2);
         Assert.IsTrue(r.CompareTo(new Rational(36, 64)) == 0);

         r = r1.Subtract(r2);
         Assert.IsTrue(r.CompareTo(new Rational(58, 64)) == 0);

         r = r1.Multiply(r2);
         Assert.IsTrue(r.CompareTo(new Rational(47 * -11, 64 * 64)) == 0);

         r = r1.Divide(r2);
         Assert.IsTrue(r.CompareTo(new Rational(47, -11)) == 0);

         r = r1.Reciprocal();
         Assert.IsTrue(r.CompareTo(new Rational(64, 47)) == 0);

         r = r1.Negate();
         Assert.IsTrue(r.CompareTo(new Rational(-47, 64)) == 0);
      }

Testing of arithmetic is a fairly straightforward idea. For all of these tests, we create a couple of rational numbers (47/64 and -11/64) and then call the various methods to perform addition, subtraction, multiplication, division, reciprocal, and negation.

The key to testing arithmetic successfully in the case of a Rational number is to know know what the result should be. As a concrete example, the result of adding these two rational numbers should be 36/64. So the testing strategy is to use the Add() method to add the two rational numbers and then test whether the result of the addition is equal to the known answer of 36/64.

As you can observe by looking at the code, the magic occurs by checking whether the computed result matches the constructed result:

Assert.IsTrue(r.CompareTo(new Rational(36, 64)) == 0);

Because we have separately tested the constructor and comparison methods, we can assume that it is ok to rely upon comparison methods as part of this arithmetic test.

And it is in this example where we begin to see the art of testing. You can write tests that assume that other tests of features you are using have already passed. In the event that your assumption is wrong, you’d be able to know that this is the case, because all of the tests you assumed to pass would not have passed.

Again, to be clear, the arithmetic tests we have done here assume that we can rely on the constructor and the comparison operation to determine equality of two rational numbers. It is entirely possible that this is not true, so we’ll be able to determine this when examining the test output (we’d see that not only the arithmetic test fails but possibly the constructor and/or comparison tests as well).

The remaining tests are fairly straightforward. We’ll more or less present them as is with minimal explanation as they are in many ways variations on the theme.

14.6. Testing Rational Conversions (to other types)¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

      [Test()]
      public void ConversionTests() {
         Rational r1 = new Rational(3, 6);
         Rational r2 = new Rational(-3, 6);
         Rational r3 = new Rational(10, -2);

         Assert.IsTrue(r1.ToDecimal() == 0.5m);
         Assert.IsTrue(r2.ToDecimal() == -0.5m);
         Assert.IsTrue(r1.ToDouble() == 0.5); //.5 is stored exactly
         Assert.IsTrue(r2.ToDouble() == -0.5);
         Assert.IsTrue(r2.ToString() == "-1/2");
         Assert.IsTrue("" + r3 == "-5");  //implicit use of ToString

      }

In this test, we want to make sure that Rational objects can be converted to floating point and decimal types (the built-in types of the C# language).

For example, Rational(3/6) is 1/2, which is 0.5 (both in its floating-point and decimal representations.

14.7. Testing the Parsing Feature¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

      [Test()]
      public void ParseTest()
      {
         Rational r;
         r = Rational.Parse("-12/30");
         Assert.IsTrue(r.CompareTo(new Rational(-12, 30)) == 0);
         r = Rational.Parse("123");
         Assert.IsTrue(r.CompareTo(new Rational(123, 1)) == 0);
         r = Rational.Parse("1.125");
         Assert.IsTrue(r.CompareTo(new Rational(9, 8)) == 0);
         Assert.IsTrue(r.ToString().Equals("9/8"));
      }

The parsing test tests whether we can convert the string representation of a rational number into an actual (reduced) rational number. We test three general cases:

• The ability to take a fraction and convert it into a rational number. This fraction may or may not have a “-” sign in it. For example -12/30 should be equivalent to constructing a Rational(-12, 30).
• The ability to take a whole number and get a proper Rational, e.g. 123 is equal to Rational(123)
• The ability to take a textual representation (1.125) and get a proper Rational(9, 8) representation. In this case, we are also getting an extra test to ensure the result is reduced.

14.8. Running the NUnit Tests¶

1. In Xamarin Studio, select the rational_nunit project.
2. In the main Xamarin Studio menu click “Run” and select “Run Unit Tests”

A test pad should appear and show something like

This likely just shows the overall results in the summary line at the bottom. You can show details by clicking on one or more of headings at the top of the pad. In particular, if you click Successful Test, Failed Tests, and Output (and likely drag the top of the pad to make it large enough to see everything), you should see something like

As you can see in the above displays, all of the tests in RationalTests get executed, and they all pass. There are no failed tests to see, but that part would be the most important details if any were there! The output just details the sequence of execution.

Xamarin Studio remembers the headings selected, so next time you run tests, the same details will show.

The testing file is using NUnit.Framework. There is a little more to this. If you edit the references, selecting All sources, and entering nunit in the search box, you see

Note there are more than one version of nunit.framework. On a Mac, the selected version worked directly, but the logical looking one, provided by Xamarin, did not work directly. Either worked if we select the context menu in the Solutions pad for the nunit.framework version added to the references, and make it look like

with the “Require Specific Version” item toggled so it is not checked.

We waited until now to discuss unit testing, because the test classes are coded with instance methods, unlike the static methods that we started out with.

A test can also call Main of a program, with specified parameters that would normally come from the command line. See the small project cmdline_to_file.