Rocky Mountain
Information Management Association

February 13, 2001, Lunch Meeting
 Click Here To Register

Finding Quality in e-business

Pete Dignan, ProtoTest



Pete Dignan

President, ProtoTest LLC

The business drivers of software quality: 

       Time to market (get it out)

       Customer satisfaction (make em happy) and

       Cost of support (dont give em a reason to call us!)

 Business Driver #1: Time-to-market

       the 92%-complete syndrome  Design, code, test, code, test, code, test.  

       Standish Group on project completion:

       Delivering software later than planned is costly whether its a product OR an internal application. 

Business Driver #2: Customer satisfaction

       Customers have more choices than ever before in most application segments. 

       Make your customers unhappy - even internal customers and you may lose them. 

       Customers expectations of software quality are up sharply in the past few years.  Customers now expect:

         Easy, quick installation




Business Driver #3: Cost of support

       Supporting a software product or an e-commerce site might have several components:

         The obvious a help desk or customer service center

         The less obvious time taken away from development, sales, others

         Also the cost of unplanned patches, service packs, custom fixes

       The cost to operate a help desk may include: 

         Staffing costs, including benefits and overhead

         Outsourced contracts

         Training costs for Help Desk staff

         Cost of any training material or promotional material distributed or made available to Help Desk customers

         Office supplies

         Cost of Help Desk Software and associated maintenance

         Depreciation or leasing fees of Help Desk hardware

         Facilities overhead (e.g. rent)

       Better product quality means fewer bugs means lower support costs.  It can also mean more effective delivery of defect information to supported customers.

How can your QA process affect these important business drivers?  Heres a brief QA and test process primer:


  1. Requirements Management

         Requirements are documented

         Requirements are reviewed for testability

         The traceability matrix is based on requirements

         Late requirements are incorporated in the process


  1. Design Reviews

         Design errors are exposed before coding

         Design elements are associated with requirements (traceability matrix)

         Testers are better able to understand system internals


  1. Code Reviews and Unit Testing

         Typically performed in Development

         Code review can be tied to traceability matrix


  1. Configuration Management

         Code control - a check-out, check -in system for source and object code

         Configuration management - includes all elements of a system, including 3rd party components


  1. Test Planning

         Master plan:  risk assessment, test types, resources, schedules

         Begins as soon as Requirements are known


  1. Test Case and Script development

         Tests are tied to requirements in the traceability matrix

         Tests are created while code is in development

         Cases and scripts are made for re-use (regression testing)


  1. Defect Tracking

         Defects are recorded, prioritized and tracked

         Team must decide which defect sources will be included

         Access to defect information can support other functions (e.g. - help desk)


  1. Test Automation

         Many types: code checking (such as for memory leaks), code coverage, commercial capture-and-playback, custom test harnesses (beyond standard de-bugging tools).

         Regression, load/stress testing are often well suited to automation

         NOT a silver bullet  (people, processes, THEN tools)


  1. Testing Metrics

         Useful in predicting schedules, defect density, defect removal efficiency etc.

         E.g. - defect origin, defect density, time-to-test per KLOC or  function point, etc.



How can such processes be applied in this new e-business world?  Here are four important trends in software and the implications for QA & Test:

Trend 1: Software is EVERYWHERE.

       10-15 years ago, software was mainly found on mainframes and minicomputers

       Now software is in your cars engine, your cell phone, your PDA; there are even IP-addressable vending machines

       Much software must run on more than one device (hardware issues), under more than one operating system (Windows, Unix, embedded), on more than one browser (IE, Netscape, AOL)

       Theres more software to test, and testing becomes MUCH more complex 

       There is a greater demand for people with software QA and testing skills than ever before

       We estimate there are around 5000 software QA and test people working in Colorado (plus or minus 20%); we could probably use another 500-1000 immediately

       Organizations need people with both test skills AND domain knowledge

Trend 2: Internet-based applications have MUCH shorter lifecycles.

       Now time-to-market is the holy grail; speed is valued over quality in many software organizations

       Many net-based businesses change the software on their site once a week some MORE than once a week

       How do we respond to still deliver a high-quality product?

1.        Become more focused on informing management of knowns, unknowns and risk

2.        Start the testing effort earlier in the development lifecycle

3.        Preventing or fixing defects earlier costs 50-200 TIMES less than later

Trend 3: Customer expectations of software quality are up sharply in the past few years. 

       Customers now expect: Easy, quick installation; Usability; Functionality; Reliability; Easy and effective support

       When they dont get those things they switch suppliers and fast

       How do companies that create software respond to these higher expectations?

1.        By using a higher ratio of testers to developers than a few years ago (trending toward 1:1)

2.        By employing testers who are more technical (black box vs. white box testing

3.        By making much more creative use of test automation (randomizing tests; data-driven testing)


Trend #4: New development methodologies are emerging, in particular eXtreme Programming, or XP.

XP involves:

       Doing the simplest thing that could work (dont over-engineer)

       Programming in pairs

       User Stories as requirements

       Writing unit tests BEFORE coding

       Refactoring extensively: combining objects and methods that are similar


How can the QA profession keep pace with changes in development process?  Working in an XP shop may require a tester to:

       Assist developers with formal development of unit tests

       Help users develop acceptance tests

       Be creative in figuring out when and how to develop and perform other types of tests that arent yet well defined in the XP methodology, like performance testing.

Web Links

An online resource for software QA and testing articles and links:

Website of the Software Quality Association in Denver:

A useful list of software QA and testing FAQs:

An in-depth look at Extreme  Programming:

This site serves the Denver/Boulder Extreme Programming  (XP) community:

This website tracks known bugs, plus fixes and workarounds, for PC software:


What do you do with  intense competitive pressure, emphasis on immediate results, and frequent resource constraints?  Pete Dignan will comment on some of the differences between traditional software testing and quality assurance in an Internet application.. He  recommends a pragmatic approach to software quality assurance.  Come, listen and debate the issues.

Pete Dignan is the founder and President of ProtoTest, a company that specializes in software quality assurance and testing services. He has seventeen years experience in the information technology field, involving system and application software, systems integration and project management. Pete has managed both product and custom software development organizations for companies such as Peak Technologies and He is a 1979 Phi Beta Kappa graduate of the University of Virginia. Pete serves on the advisory board for the CIS Masters program at the University College, University of Denver; and on the board of the QAI-affiliated Software Quality Association in Denver.

Pete Dignan President ProtoTest LLC 


RMIMA Home Page | Contact information | Board members | About RMIMA