Why TE.040 Test Scripts for UAT are a waste of time and money

I’m all for testing… I love the stuff.. but seriously, spending weeks of effort putting together a test script in the format of:

  1. Enter “Jo’s super Journal 290208” as the journal entry header
  2. Enter the journal source “Manual”
  3. Enter the journal category “Adjustment”
  4. …..
  5. ????

It seems I’ve spent weeks of my life over the last 11 years writing these things. Weeks I will never get back. (There are some nice ones uploaded by other nice humans over at http://www.oracleappsblog.com/index.php/weblog/comments/oracle-11i-test-scripts/ if you’re looking for a standard test script repository by the way).

Why I don’t like them, in a nutshell:

  • too easy to concentrate effort on the test script steps instead of taking a step back and looking at the types of testing you’re doing and what sort of coverage you’ve got across the applications, particularly with regard to negative testing and scenario-based integration testing.
  • users hate writing them and try to outsource it to the implementation partner, which is never going to work because seriously the only person who can effectively test whether you’ll still be able to do your job on Monday morning is you, not some hired gun.
  • they contribute to the intimidating thud factor. Let’s face it – would a user would like to see a 50 page test script he/she has to get through or a one page one? Keep in mind that he/she’s also juggling their actual job (which is piling up on their desk while they’re here testing this thing because no one actually back-fills the finance department)… talk about stress.
  • they get out of date very quickly and are never maintained, so when you go to do the next project you find the steps listed for ‘Enter an invoice’ are not the way Payables actually perform the task. It must be kept in line with the user procedure but there’s no method to do so.

So – back to basics – what is the purpose of User Acceptance Testing? To ascertain whether the business can continue to function with the system as delivered. Whether it be a fresh implementation, an upgrade or a really big patch, the purpose of UAT is the same.
We need to spend time, effort and money on documenting the user procedures and the training. Put the effort in where it will really be used ongoing, not just used for 2-4 weeks of UAT every six months or so.

The test scripts can then just document the actual test case or scenerio and refer out to the procedures/training guide on how to actually perform the task. If you really want a detailed, step-by-step test plan for UAT use a product to document your procedures which will produce the test script steps automatically, e.g. Oracle’s User Productivity Kit (UPK) or (my favourite) RWD’s uPerform. If you get really exited go have a look at Turnkey Solutions and see how you can have software document the testing for you apparently (haven’t tried it myself yet, but the demo looks promising). Then you can concentrate your effort on what you should be testing, rather than how you should be testing it.

5 thoughts on “Why TE.040 Test Scripts for UAT are a waste of time and money

  1. I should clarify – Turnkey’s product actually does the testing as well as documenting what it did. All the fun of Winrunner/QuickTest without the hassle of recording / re-recording / re-re-recording the scripts.
    Jo

  2. I totally agree. Writing test scripts is pain in a neck… However, I frequently warn clients against heavily relying on automated testing solutions.

    UAT is not only for testing software configurations, but it is also your last chance to test how your new BUSINESS PROCESS works with your new ERP system.

  3. Anonymous

    I agree Jo. Testing is a game of executing as many functions as possible, and the functions under test should be tested more if they have higher probability * severity of failure.

    The way to do it is to start the script authoring by listing out all the different cases as one sentence descriptions. You can easily rattle out two or three hundred cases in an hour. You should get client buy-in on your ‘lite doc’ strategy once they see how many cases there are.

    Once a script fails, it IS time to document it step by step. That you give to the developer and ask them to run it themselves before resubmitting.

    Most programs have program units that are used alot. You can find them by looking what code has the most revisions. Those will continue to require future revisions and so automated scripts are good for them.

    Also, during testing, script authoring should continue. The more bugs found in a given functional area, the more new scripts should be written for that area. Likewise, you can reduce testing in areas that show no problems. Of course as an external implementations consultant you want to know that UAT will one day come to an end, and that can be done by timeboxing and defect discovery rate counting.

    Cheers,
    Nick Carroll
    Dataweave Melbourne

  4. jim

    Your comments are spot on. I am having difficulty convincing others of the benefits of changing the way we write test scripts to what you described. can you recommend any articles, books, etc that might help (other than your blog on the topic as I plan to show them that as well!!)

  5. The problems faced by the other people are not the common problems in fact they are not even problems man the post is so clear and it defines the whole thing actually i m a student and i got CISSP certification and i know much about these things i want you to post more information…..!!!!!!!!!!

Leave a Reply

Your email address will not be published. Required fields are marked *