June 2009 - Issue #6
New environment to debug global-scale I.T. upgrades
Story by David Adams
View articles in related topics: Computing, Information Technology, Industry Collaboration
Testing new computer systems before they go live is usually standard procedure for introducing new software across a business. But when it comes to large, distributed systems such as those used by global companies, this is not an easy exercise.
While existing testing technologies can emulate, to a certain extent, enterprise-scale IT environments, they tend to be time-consuming to deploy and require considerable resources to operate. There is a lack of scalable testing environments that can efficiently and realistically imitate the complex interactions that occur among the tens of thousands of systems across big businesses on a daily basis.
“That’s the ‘it’ factor we want to achieve in this emulation environment: to emulate thousands of systems in real time with predetermined and unpredictable behaviour.”
Professor Jun Han
Professor Jun Han, who heads Swinburne University of Technology’s Software and Enterprise Systems research program, says that a typical large enterprise will have thousands of different IT systems supporting various parts of the business (from procurement to personnel management, and involving different software and hardware), each of which may need to interact with the other.
For example, he says employees often have specific clearances to access different IT systems – a set-up that is usually done with one entry through an identity management system, which will talk to all the other systems.
But the identity management system itself needs to be tested before being deployed.
“And that generally is a problem,” Professor Han says. “How do you test this software? You can’t test it in a production environment, and in a development environment you can’t set up a realistic physical testing environment running 10,000 systems.”
The existing methods for testing scenarios such as this are complex and cumbersome and there is currently no single method or tool that can efficiently test all the issues involved.
It is necessary to rely on a combination of tools and techniques, each of which tests different facets of the issues that may arise in large-scale inter-connected enterprise environments. Examples of these techniques include networks of virtual machines, load generation tools and custom scripts and software agents.
Setting up a network of virtual machines, for example, may involve running up to 100 machines and seeing whether they interact correctly.
“This may work with 100 systems but not with 1000 or 10,000 systems, in meeting quality expectations like response time and throughput,” Professor Han says.
It is a global challenge and Professor Han is now leading a collaborative research project between Swinburne and an international IT group, CA Labs, to develop a software environment that runs on just one or two computers while emulating widely distributed computing conditions and systems within a business.
Using a scalable prototype system means that the testing environment can involve as many systems as required and provide a realistic enterprise setting.
Alongside the ability to conduct functional and performance testing on a large scale, Professor Han says it is envisaged that the new emulation environment being developed would also enable other system qualities – such as how the system responds to delay or to unpredictable behaviour – to be more accurately tested than has previously been possible.
“That’s the ‘it’ factor we want to achieve in this emulation environment: to emulate thousands of systems in real time with predetermined and unpredictable behaviour. It’s about quality assurance (QA), knowing that a new system will function properly, no matter how many systems it needs to talk to and even if some of these systems are not available or down.”
The project is being funded by CA Labs, which was established by global software firm CA in 2005 to strengthen relationships between the company and research communities. It has been running for two years and is expected to continue beyond its initial three-year timetable.
As well as Professor Han, the team of Swinburne researchers working on the project includes Dr Jean-Guy Schneider, Dr Lars Grunske and PhD student Cameron Hine.
The project is based on, and further extends, Swinburne’s research expertise in system integration, software architecture, and system performance and reliability.
A prototype system, known as REACTO, was demonstrated at the four-day CA World conference in Las Vegas in November 2008. The demonstration emulated 10,000 enterprise systems successfully interacting with CA’s ‘Identity Manager’ software as the system under test.
Dr Steve Versteeg, a research staff member at CA Labs in Melbourne and also a member of the project team, says the concept has already attracted considerable interest within CA and its customer base, which represents many of the world’s largest enterprises.
Noting that while all CA software is already comprehensively tested before it goes out, Dr Versteeg adds that the scalable emulation project would further enhance that.
“QA has existing methods for testing the scalability of a system,” he notes. “What this can do is to unify that and simplify and extend it.”
