Guild Wars 2

Hello forum

If I’m not mistaken, the APIs we’re given are designed to be consumed in loosely typed languages. Nevertheless, we successfully wrote a .NET library that wraps the API in strongly typed classes.

The problem is that I’m not exactly sure how to go about unit testing code that relies on external data.

For APIs that do not require any parameters, I can simply get the data once and use that to verify that my code behaves as expected.

As an example, I test my code for the ‘items.json’ API like this:

    [Test]
    public void GetItemsCatalog()
    {
        var request  = new ItemsRequest();
        var response = request.GetResponse(client);

        Assume.That(response.IsSuccessStatusCode);
        Assume.That(response.IsJsonResponse);

        var itemsResult = response.Deserialize();

        Assert.IsNotNull(itemsResult);
        Assert.IsEmpty(itemsResult.ExtensionData, "The '{0}' class is missing one or more properties.", typeof(ItemsResult).FullName);

        Assert.IsNotNull(itemsResult.Items);
        Assert.IsNotEmpty(itemsResult.Items);
        Assert.IsEmpty(itemsResult.Items.ExtensionData, "The '{0}' class is missing one or more properties.", typeof(Items).FullName);

        Trace.WriteLine(string.Format("Number of items: {0}", itemsResult.Items.Count));
    }

Although this unit test assumes that my machine has a working internet connection (and that the web service is up), its test results are fairly accurate. Either my code behaves as intended, or it doesn’t.

But what do I test for when an API’s response data changes over time due to the dynamic nature of the game?

At the time of writing, there are 31,864 unique item IDs that can be used to query the ‘item_details.json’ API. In order to say with 100% certainty that my ItemDetails class behaves as expected, I would have to test my code with 31,864 different inputs.

Obviously, this is a no-go. So how do I test this code?

If I’m not mistaken, the APIs we’re given are designed to be consumed in loosely typed languages.

You are mistaken. JSON is just a data exchange format which describes values. It does nothing to impose a specific “language” meant to consume it (yes, I realize that it originated from JavaScript). If JSON wasn’t as flexible as it is, and it was so restrictive as to make it difficult for developers of, say, statically type languages, to consume, it would not be so widely used.

If a client application consumes JSON, and, for example, a value for ‘someID’ is passed down as a string, then if a programmer decides to convert that string value to an unsigned integer, so be it. That’s a architectural design decision, but this has nothing to do with the JSON concept or the provider of the APIs. If it works, it works.

The problem is that I’m not exactly sure how to go about unit testing code that relies on external data.

Unit tests should be self-sustaining. You shouldn’t bake external dependencies within your tests, if you can help it. That that I recently solved this was with a testing concept called ‘fixtures’. A fixture can be considered pre-determined input passed into the unit test which mirror real-world data.

Note: unit testing is a broad topic. Here, fixtures are present as just one possible solution. There are other alternatives, such as mock objects.

I recent created a development branch in GW2KIt which adds a plethora of unit tests. These tests use fixtures I created from version 1 (v1) of the APIs.

Again, look here for list of v1 fixtures I use.

Edit

But what do I test for when an API’s response data changes over time due to the dynamic nature of the game?

You do your best to mitigate issues. “Doing your best” means combing the documentation, in this case item_details, and writing tests based on what’s described there. You might want to try making a fixture for one of each type of item, and running it through your unit test.

If you’ve done your job correctly, you’ll be pretty well set. Then, it’s up to the API’s creators to communicate with the community about possible breaking changes. (Spoiler Alert: things will be changing soon).

Sorry, I wasn’t referring to the JSON’s response format. I remembered one of the devs talking about how they designed the APIs.

Cliff Spradlin.3512:

I don’t have much experience using JSON libraries that map to strongly-typed objects. We didn’t really design the API in mind with that use-case, so I’m not surprised that there are places where it breaks or doesn’t make sense.

So anyway, a test fixture is nothing but a vague description of a unit test that uses static sample data from an otherwise dynamic data source as input? I think that gave me an idea.

So anyway, a test fixture is nothing but a vague description of a unit test that uses static sample data from an otherwise dynamic data source as input? I think that gave me an idea.

A fixture is the static sample data.

Alright, I think I get it now.

My idea is to write individual tests for each ItemDetails property. One test to check that the target property isn’t null, and for properties where the range of allowed values is finite, one additional null-test per known value (e.g. ‘item_details.type’, ‘item_details.rarity’).

Would that be a meaningful way to test?

It’s quite easy — https://github.com/jsumners/gw2-api-client/blob/develop/src/test/java/com/jrfom/gw2/ItemDetailsTest.java

One solution we use with other projects is to run the tests, and the first time it will actually pull the data from the server and save this to the system. Any tests after this will just run tests on the downloaded JSON data (so only connects to the internet once, and once only).

And then if you want to download the data again, we just delete the JSON files, and the tests automatically connect to the API again to re-pull data (in case anything may have changed, to check updated behaviour and such).

I took a slightly different approach. Rather than testing a particular response, I turn it into a JSON schema by replacing every value with a generic substitute.

numeric : 0
text : ""
collection : []
complex type: {}

With that as the input, all I have to do is

1. check that every object property is initialized to its default
2. check that the serializer could find a matching property on the .NET class for every property in the JSON input.

Example: https://gw2dotnet.codeplex.com/SourceControl/latest#GW2.NET-Http/GW2.NET Tests/Core/ItemsInformation/Details/ItemTest.cs

First here is the correct link Steven posted:

http://gw2dotnet.codeplex.com/SourceControl/latest#GW2.NET-Http/GW2.NET Tests/V1/Core/Items/Details/ItemTest.cs

Another approach which I used in the early days of “GW2.NET”: https://gw2dotnet.codeplex.com/ was the following:

• We know from the wiki which response to expect.
• You fetch the data which is preseted at the wiki and write your tests to compare it
• If your fetched data is equal to the data you have in the wiki (hardcode it into your tests or something) you know your code works.

Another approach, since querying some nodes (e.g. items.json) is time consuming, is to use mocking. Basically you pass the data to your method you know the api is going to return (in this case a JSON formatted string) and then work with that on your code. This is usually better than querying the api every time, since we expect the api to return data and it is not your responsibility to unit test the ANet api. So here is what you would do:

• Take the data the api is going to return (from the api itself or otherwise) and pass it to your code via a mocking framework.
• Test your code with unit tests and the mocking framework.

I personally recommend Moq as mocking framework and NUnit as testing framework. I have made great experience with both. But this is a personal preference use what ever you like.