I was just giving my last post some more thought whilst working on a test which actually is in a UAT environment, and a few more things came to mind which I didn’t mention….
Firstly, there is of course not a lot of point in doing any kind of infrastructure testing as part of the web test in a non-production environment, unless the customer is 100% sure that the UAT environment is identical to the production one. In my experience, few UAT environments are exactly the same as live (particularly in non-enterprise companies), and there is not a whole lot of point in the tester wasting time writing up a load of findings about SSL issues which exist in UAT simply because it isn’t intended to be accessed by real customers. This isn’t to say that companies shouldn’t make their test environments as similar as production as possible however, just that it makes sense to cut a little slack provided the issue doesn’t really exist in production.
Secondly, if you are providing a test environment for your security review, you won’t get the most out of it if you don’t make sure that everything actually works properly. By which I mean that it is quite common to be given a test application where key parts of the functionality are non-functional, and this is particularly common whenever an ancillary system is needed (forgotten passwords which rely on email are notorious for not working in test, as are anti-malware controls). Another thing to think about for sites involving financial transactions, is how to provide this functionality to the tester. Generally the best approach here is to stub the transactional functionality out at the point where in production the purchase would be passed to the payment provider. Finally, it is really important that the test site contains some meaningful data – it is pretty useless to be given an application with an empty database.
So the best of all worlds, and what produces the best test results, is to have a fully functional site in a test environment for the application testing, and then to test the infrastructure components on the corresponding servers in live.