Consequences of array covariance in C#

Posted on Fri 22 October 2021 in Coding

In today’s post, I’ll talk about array covariance in C#, how it hid a bug in code that was covered by a passing unit test (*gasp*), and some ideas for how to avoid such bugs.

What is covariance?

In computer science, variance has to do with how complex types relate to each other based on the relation between their components. For example, consider the two types Car and Vehicle. A car is a vehicle, so the relationship can be expressed in code as:

class Car : Vehicle
{
    ...
}

Now that there is a relation between the two types—Car is a subtype of Vehicle—what is the relation between the following types?

  • IList<Car> and IList<Vehicle>?
  • Action<Car> and Action<Vehicle>?

This is where variance rules come in—they define how such types relate.

Covariance, then, means that two complex types are related to each other in the same way as their components. We’ll come back to the built-in .NET types in a moment, but for now let’s consider a custom interface, IParkingSpace<T>. If this interface is covariant on T, then IParkingSpace<Car> is a subtype of IParkingSpace<Vehicle>, because Car is a subtype of Vehicle. Put differently, parking space covariance means that if you have a parking space for cars, you can say that it is a parking space for vehicles. As you can imagine, whether or not this holds true in real life depends very much on what you intend to do with the parking space. Saying “yes, a vehicle is parked” is fine, but “let’s park an arbitrary vehicle” will likely not turn out well.

Covariance on a type parameter is declared as follows:

interface IParkingSpace<out T>

One might ask why the keyword to specify covariance is out. How does that make sense? The answer is that is has to do with how the type T is used inside the interface. If T only occurs as the return type of members of the interface, then T only occurs “on the way out”:

                      +------------+
    Arguments in ---> |   method   | --> Return value out
                      +------------+

This means that we can have:

interface IParkingSpace<out T>
{
    T GetCurrentlyParked();
}

But we cannot have:

interface IParkingSpace<out T>
{
    // THIS DOES NOT COMPILE!
    void Park(T parkee);
}

Since we have specified out T to indicate covariance on type T, we cannot use T as an input parameter, only as a return type.

Side note: Confusingly, covariant T cannot appears as an out parameter of a method. For example, void GetCurrentlyParked(out T parked) doesn’t compile.

Side note 2: If the return type of a member is a complex type, then that complex type must also be covariant on T.

Ok, but so far we have at best specified a mnemonic for covariance being declared using out. Why is it important that covariance requires T to appear as return type only?

To understand that, consider a non-variant version of the type IParkingSpace<T>:

interface IParkingSpace<T>
{
    T GetCurrentlyParked();
    void Park(T parkee);
}

Since T isn’t qualified with out, we can use it both as a return type and as a parameter type. Then consider this piece of incorrect code:

IParkingSpace<Car> parkingSpaceForCars = new ParkingSpace<Car>();
IParkingSpace<Vehicle> parkingSpaceForAnyVehicle =
    (IParkingSpace<Vehicle>) parkingSpaceForCars;

parkingSpaceForAnyVehicle.Park(new FreightTrain()); // FreightTrain is a Vehicle

Do you see the problem? We cast the parking space for cars to a parking space that accepts any vehicle, and then try to park a freight train there. Just as in real life, that won’t end well!

It turns out that the code won’t even execute successfully. The cast in the second statement will fail and throw an InvalidCastException, thereby preventing the illegal Park call, precisely because the interface in this example is non-variant.

Since compile-time errors are infinitely better than runtime errors, we’d like to have a mechanism that allows the compiler to prevent us from making mistakes like the above. This is why declaring covariance on a complex type is useful!

In the example above, if we declare IParkingSpace to be covariant on T, the compiler will prevent us from adding the Park method, and furthermore the cast from IParkingSpace<Car> to IParkingSpace<Vehicle> will succeed.

.NET collection type covariance

In .NET we have two commonly used collection types, IList<T> and IEnumerable<T>. The former is read/write and the latter is read-only. Since IList<T> is read/write, it follows that T must appear as both return type and parameter type of methods in the interface, and that the interface therefore cannot be covariant on T.

We can confirm that IList<T> is not covariant on T:

IList<Car> cars = new List<Car>();
IList<Vehicle> vehicles = cars as IList<Vehicle>;
Console.WriteLine($"Is IList<T> covariant? {vehicles != null}");

(Remember, a direct cast throws InvalidCastException for an invalid cast, whereas the as operator returns null.)

If you run the snippet above, it will print:

Is IList<T> covariant? False

Similarly, we can confirm that IEnumerable<T> is covariant on T:

IEnumerable<Car> cars = new List<Car>();
IEnumerable<Vehicle> vehicles = cars as IEnumerable<Vehicle>;
Console.WriteLine($"Is IEnumerable<T> covariant? {vehicles != null}");

This snippet will print:

Is IEnumerable<T> covariant? True

Great! Everything seems to work as expected. Next up, array covariance!

Array covariance

In C#, an array T[] is covariant on T, except for value types. You can read up on the details here, but for now let’s consider an example similar to the previous ones:

Car[] cars = new Car[1];
Vehicle[] vehicles = cars as Vehicle[];

Console.WriteLine($"Is T[] covariant? {vehicles != null}");

This snippet will print:

Is T[] covariant? True

Fantastic! Arrays behave just like IEnumerable<T> in this case, it appears.

So, why does C# have array covariance? Well, let’s say that we want to write a function that counts the number of non-null objects in an array, but we live in the past where generics have not yet been introduced in the language. What would the signature of such a function look like? To avoid having to create an infinite number of overloads, it makes sense to go for:

static int CountNonNullObjects(object[] objects)

Thanks to array covariance, it’s possible to pass almost any kind of array to the function, for example Car[]. Again, array covariance doesn’t apply to value types, so it won’t be possible to pass int[], for example.

So, at this point we know that IList<T> is not covariant on T, but T[] is. But wait a minute—doesn’t an array T[] implement IList<T>??

T[] implements IList<T>

Yes, yes it does! We can write:

IList<Car> cars = new Car[1];

Now, this is interesting from a covariance perspective. What happens if we run the following?

IList<Car> cars = new Car[1];
IList<Vehicle> vehicles = cars as IList<Vehicle>;
Console.WriteLine($"Is T[] disguised as IList<T> covariant? {vehicles != null}");

The result is:

Is T[] disguised as IList<T> covariant? True

In other words, the fact that an array is covariant takes precedence over the fact that IList<T> isn’t. This is a bit problematic, because it means that the statement IList<T> is not covariant on T is demonstrably false—it depends on the list implementation.

And this, dear reader, is the source of the aforementioned bug.

The bug

The bug was introduced in a proxy function, initially similar to the following:

IList<Vehicle> GetVehicles()
{
    Configuration configuration = GetConfiguration();
    IList<Vehicle> vehicles = configuration.Vehicles;
    return vehicles;
}

An important detail is how Configuration was defined:

class Configuration
{
    ...
    IList<Vehicle> Vehicles { get; set; }
    ...
}

One of the test cases was similar to this:

[Test]
void Should_return_vehicles()
{
    Configuration configuration = new Configuration
    {
        Vehicles = new Vehicle[] { Volvo, Tesla }
    };
    VehicleProxy proxy = new VehicleProxy(configuration);

    IList<string> makes = proxy.GetVehicles().Select(v => v.Name).ToList();

    Assert.That(makes, Is.EqualTo(new string[] { "Volvo", "Tesla" }));
}

At some point, there was a need to change the signature of Configuration.Vehicles so that it was a list of Cars instead of Vehicles:

class Configuration
{
    ...
    IList<Car> Vehicles { get; set; }
    ...
}

The test case was updated accordingly:

[Test]
void Should_return_vehicles()
{
    Configuration configuration = new Configuration
    {
        Vehicles = new Car[] { Volvo, Tesla } // <== change initialization here
    };
    VehicleProxy proxy = new VehicleProxy(configuration);

    IList<string> makes = proxy.GetVehicles().Select(v => v.Name).ToList();

    Assert.That(makes, Is.EqualTo(new string[] { "Volvo", "Tesla" }));
}

And finally, the proxy function was updated:

IList<Vehicle> GetVehicles()
{
    Configuration configuration = GetConfiguration();
    IList<Vehicle> vehicles =
        configuration.Vehicles as IList<Vehicle>; // <== add cast here
    return vehicles;
}

Do you see the problem? The test case didn’t—it was as green as it could ever be!

As we have seen before, IList<T> is not covariant, so the cast to IList<Vehicle> is simply not valid. When the code was run in production, the as operator returned null, and we got a NullReferenceException downstream.

But why didn’t the test case detect the problem? It’s because of this property initialization:

Vehicles = new Car[] { Volvo, Tesla }

Since arrays are covariant even when treated as IList<T>, the cast succeeded when GetVehicles was excercised by the test case. However, in the real code, the property was initialized using a List<Car> instance, in which case the non-variance of IList<T> applied and the cast failed.

Lessons learned

Avoid IList<T> casting

In general, casting that involves IList<T> must be carefully scrutinized.

Unless there are special performance and/or memory requirements, using the LINQ method Cast is a lot safer:

IList<Vehicle> vehicles = configuration.Vehicles.Cast<Vehicle>().ToList();

OfType is an option as well; it depends on what you know about the element types.

Use good property types

Using type IList<...> for a property has a number of problems, for example that the caller can grab the list and mutate it. To prevent that, you have to ensure the returned list is read-only.

A much better option is to use IEnumerable<...>, which is both read-only in itself and covariant on the element type—and thus safe to cast.

Enable nullable references?

Since the as operator may return null, it can be argued that the function above should be changed further:

#nullable enable               // <== enable nullable reference types
IList<Vehicle>? GetVehicles()  // <== add ? to the return type
{
    Configuration configuration = GetConfiguration();
    IList<Vehicle> vehicles = configuration.Vehicles as IList<Vehicle>;
    return vehicles;
}

While this makes the code more correct, it wouldn’t have helped in our case. The use of a nullable reference type would have propagated downstream, and at some point null would likely have been converted to an empty list, so we would still have a functional breakage (albeit an exception-less one).

Test case setup must be true to the production code setup

This is the most important lesson learned!

You may have thought about this in the context of mocks, stubs and fakes. If you use a mock with a behavior that differs even the slightest from the code it mimics, the usefulness of your tests will suffer.

As we can see here, the principle is also important when it comes to choosing the implementation type for an interface. Had the test case used List<Car>, the problem would have been spotted immediately.

csharp covariance cast