Evolution of a Test Method

The development of Eurotunnel's commercial information systems prior to the tunnel's opening presented a number of unusual testing challenges:

• Rapidly evolving requirements within a new organisation, requiring fast-track development as the requirements became known,
• No existing business, procedures or systems with which to compare the new systems,
• Huge project and business creation pressures severely limiting user time and their ability to participate in system development,
• Rapid delivery of multiple external systems concurrently with the final physical infrastructure,
• A variety of system architectures, histories and underlying quality levels,
• Use of then new technologies, such as 4GLs, client/server structures and graphical interfaces.

Traditional testing methods didn't adequately address these problems, so it was necessary to develop a new test method which was practical, matched to the target environments and systems, and was realistic about what could be achieved. This method developed steadily with a number of major milestones:

1. The establishment of a "building block" principle tying the major stages of testing into the stages of Eurotunnel's main development method. This was vital for understanding of the structure and principles of testing, and in particular the reliance of later test stages on earlier testing.

2. The development of relevant and effective test techniques for 4GL developments. The formal methods for unit testing and inspection were derived from proven 3GL techniques, but extended and made more practical for the complexities of 4GL code.

3. The creation of teams focusing Information Services and user effort around the testing of a number of key systems. Initially, "full system tests" made sure that the systems met their basic technical and functional requirements and that the users were trained in their use.

4. These tests then became the "Full Operational Trials", or FOTs. The FOTs were simulations of live operation in as realistic an environment as possible. As well as a mechanism for successfully involving developers and users with widely different objectives and sometimes conflicting priorities, it allowed users to develop their procedures in a non-threatening environment, including those for "what do we do when the system doesn't work?". The company could therefore make policy, organisational and system changes before starting real business operations.

There were some difficulties - early attempts to use tools to automate testing failed because the systems were not yet adequately stable, and some of the test methods proved unworkable for externally-produced systems with less formal development histories. 

However, overall the outcome was a successful one: Eurotunnel had systems which met the initial demands as an operating company, and we knew where the weak spots were as a result of structured testing, so we could effectively manage around these deficiencies.

This presentation, made to EuroSTAR '94, describes the evolution of the test method, roughly in parallel with the status of the major systems.