Setting Up Tests

Test setup in Spatineo Performance is done in two phases: At first, a test assistant dialog guides you through the basic test setup selections. Then you can fine-tune the test parameters and add new meters according to your wishes.

Test Setup Assistant

The Test assistant dialog opens up automatically when entering Spatineo Performance. The assistant will guide you through setting up a new test using one of the three pre-defined test profiles (see Table 1). If the service you’re about to test is not yet claimed by your organisation, you are give instructions on how to do that by the assistant. When the assistant is done, you have a complete setup ready for running typical performance test. If you wish you can also change any of the test parameters created for you by the assistant, add or remove meters to be tested, or change the test length and ramp-up time. Any changes you make are saved automatically. If you want to open a previously created test instead, press cancel button and select the test using the “Existing tests..” button.

Table 1: Pre-defined test profiles for WMS, WMTS and WFS services

If the Service Cannot Be Found

In case you cannot find the service you want to test using the search provided by the Test setup assistant, it’s a good idea to check that the service is registered in our service catalog. It may happen that we do not yet know about a service that is new or not added to well known catalog services and portals.

It’s simple to check if a service is known to us: just copy-paste the complete URL address of the get capabilities request for the given service to the search field of Spatineo Directory available at http://directory.spatineo.com/ and hit Enter. We’ll start analysing the URL right away and add in to our catalog if we did not know about it before. Note that this simple method only works for services not requiring user authentication. If your service is password protected or requires some other user authentication please contact our support to set the access settings.

Use Spatineo Directory to check if a service is known to us, and thus
selectable for testing. You can check this for any service publicly available in the Internet
by copy-pasting the URL address of the service endpoint in the search box.

Selecting Meters For the Test

A meter is a specific way of calling a service: It contains the service identification and the operation to execute as well as all the parameters fixed parameters, like the layer or feature type, image size and type and map projection to use. Spatineo Performance automatically varies the bounding box (area) of the request during the tests in order to make the test more realistic.

Service type	Types of meter supported
Web Map Service	GetMap GetCapabilities
Web Map Tile Service	GetTile GetCapabilities
Web Feature Service	GetFeature DescribeFeatureType GetCapabilities
ArcGIS MapServer	export service description retrieval

For each test you can add as many meters as you like for one or more services that you have claimed ownership for. You can define the maximum request rate separately for each meter. This enables you create complex test scenarios like testing both WMS and WFS interfaces of the same server at the same time or generating predefined loads for services sharing same backend resources or network bandwidth limits. Note however that adding too many meters for the same test can make it more difficult for you to analyse the test results. The total request rate for all the meters in one test is also limited to 1000 requests/s by default. Please contact our support if you want to run tests with higher request rates.

Some layers or features are typically more demanding for a service to produce than others, thus it’s often a good idea to create tests for different types of offered data. Sometimes using certain map projections can also be more demanding to the server products than others because of the necessary on-the-fly re-projection calculations.

Advanced meters

By default, meters accessing data on the service (for example GetMap for WMS services) will request data within the bounds declared in the service description. However, sometimes you might require a test where a meter focuses on a smaller area, or an altogether different area. This can be achieved by opening "Additional options" in the performance meter and setting a bounding box.

Limiting a performance meter to a particular bounding box

You can consult the table below to see how the bounding box values are translated into requests for each request type.

Service and request type	Coordinate order	Other notes
WMS/GetMap WFS/GetFeature MapServer/export	Regardless of projection axis order, BBOX parameter values will use X values as the first coordinate and Y as the second.	X and Y are flipped if the selected projection is configured for "flip axis order"
WMTS/GetTile	Given X and Y coordinates are used in the projection axis order (X as first coordinate) to calculate tile indices. Axis flip configuration has no bearing on how GetTile requests are generated.	The meter will target only those matrices and tiles which overlap both the given bounding box and the matrix bounds taking into account any TileMatrixLimits configuration. If the given bounding box is outside the legal bounds of all tile matrices for this projection, the test will fail at the prepare stage.
MapServer/tile	Given X and Y coordinates are used in the projection axis order (X is first, Y is second) to calculate tile indices. Axis flip configuration has no bearing on how GetTile requests are generated.	If given bounds are outside the tiled area, Spatineo Performance may produce illegal tile requests.

Running the Same or Similar Test Again

If you want to re-run a previously ran test or use it’s configuration as a basis for your new test you can use the “Re-run test” button at the Setup tab. It creates a new test with the same settings as in currently selected test. You can give the new test a describing name and modify the test parameters as desired. The original test remains archived in Spatineo Performance so you can return to analyse it at any time.

Running and Reviewing Previously Created Tests

All test created by your organisation are stored in the test archive regardless of their completion status. You can design a number of tests in advance and just run then one ofter another during a testing session. Another typical use case would be iterative testing, where you want to start with a basic test settings and modify them in the latter tests to try to expose undesired service behaviour or find the service capacity limits.

You can select any of the previously created tests by clicking the “Existing tests..” button. The opening dialog lets you choose a test and delete tests that you no longer consider worth keeping. You can change the viewed test at any time, even when the test is running. All the changes you made to the test are continuously and automatically stored.