Testing in Live Environments

Apr 12th, 2016

Comments: 0
Category: Uncategorized

Testing in Live Environments

It seems pretty obvious that carrying out web application testing in a live production environment is in general a bad idea. Exactly how bad an idea depends on the nature of the site, but considering that many testing functions involve deliberately creating data which is either badly formatted, or is in a location where it is not supposed to be, it should be clearly explained to customer from the outset that this is not something they want going on in their live site unless it is absolutely unavoidable.

Just to give a few examples of where live testing could be disastrous….

You have a site where the tester should only be able to create data in his own designated area. But there is an authorisation flaw on it which makes it possible for data to be created under other users’ areas. The tester has run a scanner and all of a sudden you have a situation where all your clients have test data popping up under their own accounts. Or the site has cross-site scripting errors and now all of a sudden your sales users are getting JavaScript popup boxes every time they try to go to their admin pages. Or the site has SQL injection and by effectively running an SQL query blind, the tester amends or deletes every record in the database (or manages to drop the database and take the site down). Or (and I have seen this happen) a tester manages to upload a shell to a live server and then can’t delete it again, so it just gets left there – ending up making a site that was insecure to start with a whole lot worse.

So we always carefully explain this before we start, and many customers sensibly chose to let us test in a UAT environment instead. Of course, even in these days of VMs and cloud services, there are times when it is not possible to set up a test environment (perhaps due to timing constraints), and this doesn’t make it impossible to test, just a whole lot more difficult. Given a production environment, the tester should not be making extensive use of scanners or any automated tool where he can’t absolutely guarantee that he knows what data it is creating, where it is being created, and how to get rid of it afterwards. In some environments it may not be possible to create data at all, and if this is the case, some auxiliary method needs to be used to create assurance that critical issues are not present – this is yet another good reason for having a copy of the backend code available.

In any case, no purchaser of testing services should end up paying for an application test in a live environment without fully understanding the consequences of this and what his other options are, and it is the responsibility of the company selling the test to make this clear up front.

Add a comment

Your email address will not be shared or published. Required fields are marked *