Fluent Essays

Read the article carefully and then answer the 2 questions Defining software quality To begin this course, this reading introduces you to the broader ideas behind the topic of software quality and provides some initial perspectives on why quality is hard to measure, and how software engineers have approached this tough measurement problem. You will also learn that there is no single, universal definition of software quality, and the concept is used with varying meanings in different situations. To read Each week of the semester, you will have a reading assignment to complete that includes a brief written response to two questions based on the reading. • [required] Robert Glass. The link between software quality and software maintenance . Software Conflict: Essays on the Art and Science of Software Engineering, Prentice-Hall, 1991, pp. 49-51. • [required] Robert Glass. Can you MANAGE quality into a software product? Software Conflict: Essays on the Art and Science of Software Engineering, Prentice-Hall, 1991, pp. 147-149. • [required] Roger S. Pressman. Chapter 14: Quality Concepts . Software Engineering: A Practitioners Approach, 7th ed., McGraw-Hill, 2010, pp. 398-415 [can also use 8th edition]. To turn in Prepare a brief (no more than one page) written answer to the following two questions. Write up your answer using MS Word or LaTex. One well- presented paragraph for each question is sufficient. 1. Robert Glass does not include all of the quality attributes that are in McCalls list of quality factors. Do you believe the extra attributes in McCalls list are necessary for quality software, or is Glass definition sufficient? Justify your answer. 2. In Glass definition, which single attribute do you believe is most important, and why? Defining software quality To read To turn in The drumbeat for improved software quality began in earnest as softwarebecame increasingly integrated in every facet of our lives. By the 1990s,major corporations recognized that billions of dollars each year were be- ing wasted on software that didn’t deliver the features and functionality that were promised. Worse, both government and industry became increasingly concerned that a major software fault might cripple important infrastructure, costing tens of billions more. By the turn of the century, CIO Magazine [Lev01] trumpeted the headline, “Let’s Stop Wasting $78 Billion a Year,” lamenting the fact that “American businesses spend billions for software that doesn’t do what it’s sup- posed to do.” InformationWeek [Ric01] echoed the same concern: Despite good intentions, defective code remains the hobgoblin of the software indus- try, accounting for as much as 45\% of computer-system downtime and costing U.S. companies about $100 billion last year in lost productivity and repairs, says the Standish Group, a market research firm. That doesn’t include the cost of losing angry customers. Because IT shops write applications that rely on packaged infrastructure software, bad code can wreak havoc on custom apps as well. . . . Just how bad is bad software? Definitions vary, but experts say it takes only three or four defects per 1,000 lines of code to make a program perform poorly. Factor in that most programmers inject about one error for every 10 lines of code they write, multiply that by the millions of lines of code in many commercial products, then figure it costs software vendors at least half their development budgets to fix errors while testing. Get the picture? 398 C H A P T E R 14 QUALITYCONCEPTS K E Y C O N C E P T S cost of quality . .407 good enough . . .406 liability . . . . . .410 management actions . . . . . . .411 quality . . . . . . .399 quality dilemma . . . . . .406 quality dimensions . . . .401 quality factors .402 quantitative view . . . . . . . .405 risks . . . . . . . .409 security . . . . . .410 What is it? The answer isn’t as easy as you might think. You know quality when you see it, and yet, it can be an elusive thing to define. But for com- puter software, quality is something that we must define, and that’s what I’ll do in this chapter. Who does it? Everyone—software engineers, managers, all stakeholders—involved in the software process is responsible for quality. Why is it important? You can do it right, or you can do it over again. If a software team stresses quality in all software engineering activities, it reduces the amount of rework that it must do. That results in lower costs, and more importantly, improved time-to-market. Q U I C K L O O K What are the steps? To achieve high-quality software, four activities must occur: proven soft- ware engineering process and practice, solid project management, comprehensive quality control, and the presence of a quality assurance infrastructure. What is the work product? Software that meets its customer’s needs, performs accurately and reliably, and provides value to all who use it. How do I ensure that I’ve done it right? Track quality by examining the results of all quality control activities, and measure quality by exam- ining errors before delivery and defects released to the field. pre75977_ch14.qxd 11/27/08 5:51 PM Page 398 C H A P T E R 1 4 Q U A L I T Y C O N C E P T S 399 In 2005, ComputerWorld [Hil05] lamented that “bad software plagues nearly every organization that uses computers, causing lost work hours during computer down- time, lost or corrupted data, missed sales opportunities, high IT support and mainte- nance costs, and low customer satisfaction. A year later, InfoWorld [Fos06] wrote about the “the sorry state of software quality” reporting that the quality problem had not gotten any better. Today, software quality remains an issue, but who is to blame? Customers blame developers, arguing that sloppy practices lead to low-quality software. Developers blame customers (and other stakeholders), arguing that irrational delivery dates and a continuing stream of changes force them to deliver software before it has been fully validated. Who’s right? Both—and that’s the problem. In this chapter, I consider soft- ware quality as a concept and examine why it’s worthy of serious consideration whenever software engineering practices are applied. 1 4 . 1 W H AT I S Q U A L I T Y ? In his mystical book, Zen and the Art of Motorcycle Maintenance, Robert Persig [Per74] commented on the thing we call quality: Quality . . . you know what it is, yet you don’t know what it is. But that’s self-contradictory. But some things are better than others; that is, they have more quality. But when you try to say what the quality is, apart from the things that have it, it all goes poof! There’s noth- ing to talk about. But if you can’t say what Quality is, how do you know what it is, or how do you know that it even exists? If no one knows what it is, then for all practical purposes it doesn’t exist at all. But for all practical purposes it really does exist. What else are the grades based on? Why else would people pay fortunes for some things and throw others in the trash pile? Obviously some things are better than others . . . but what’s the better- ness? . . . So round and round you go, spinning mental wheels and nowhere finding any- place to get traction. What the hell is Quality? What is it? Indeed—what is it? At a somewhat more pragmatic level, David Garvin [Gar84] of the Harvard Busi- ness School suggests that “quality is a complex and multifaceted concept” that can be described from five different points of view. The transcendental view argues (like Persig) that quality is something that you immediately recognize, but cannot explic- itly define. The user view sees quality in terms of an end user’s specific goals. If a product meets those goals, it exhibits quality. The manufacturer’s view defines qual- ity in terms of the original specification of the product. If the product conforms to the spec, it exhibits quality. The product view suggests that quality can be tied to inher- ent characteristics (e.g., functions and features) of a product. Finally, the value-based view measures quality based on how much a customer is willing to pay for a prod- uct. In reality, quality encompasses all of these views and more. Quality of design refers to the characteristics that designers specify for a product. The grade of materials, tolerances, and performance specifications all contribute to the quality of design. As higher-grade materials are used, tighter tolerances and What are the different ways in which quality can be viewed? pre75977_ch14.qxd 11/27/08 5:51 PM Page 399 greater levels of performance are specified, the design quality of a product increases, if the product is manufactured according to specifications. In software development, quality of design encompasses the degree to which the design meets the functions and features specified in the requirements model. Quality of conformance focuses on the degree to which the implementation follows the design and the resulting system meets its requirements and performance goals. But are quality of design and quality of conformance the only issues that software engineers must consider? Robert Glass [Gla98] argues that a more “intuitive” rela- tionship is in order: user satisfaction ! compliant product good quality delivery within budget and schedule At the bottom line, Glass contends that quality is important, but if the user isn’t satisfied, nothing else really matters. DeMarco [DeM98] reinforces this view when he states: “A product’s quality is a function of how much it changes the world for the better.” This view of quality contends that if a software product provides substantial benefit to its end users, they may be willing to tolerate occasional reliability or per- formance problems. 1 4 . 2 S O F T WA R E Q U A L I T Y Even the most jaded software developers will agree that high-quality software is an important goal. But how do we define software quality? In the most general sense, software quality can be defined1 as: An effective software process applied in a manner that creates a useful product that provides measurable value for those who produce it and those who use it. There is little question that the preceding definition could be modified or extended and debated endlessly. For the purposes of this book, the definition serves to emphasize three important points: 1. An effective software process establishes the infrastructure that supports any effort at building a high-quality software product. The management aspects of process create the checks and balances that help avoid project chaos—a key contributor to poor quality. Software engineering practices allow the developer to analyze the problem and design a solid solution—both critical to building high-quality software. Finally, umbrella activities such as change management and technical reviews have as much to do with quality as any other part of software engineering practice. 2. A useful product delivers the content, functions, and features that the end user desires, but as important, it delivers these assets in a reliable, error-free 400 P A R T T H R E E Q U A L I T Y M A N A G E M E N T uote: “People forget how fast you did a job— but they always remember how well you did it.” Howard Newton 1 This definition has been adapted from [Bes04] and replaces a more manufacturing-oriented view presented in earlier editions of this book. pre75977_ch14.qxd 11/27/08 5:51 PM Page 400 way. A useful product always satisfies those requirements that have been explicitly stated by stakeholders. In addition, it satisfies a set of implicit requirements (e.g., ease of use) that are expected of all high-quality software. 3. By adding value for both the producer and user of a software product, high- quality software provides benefits for the software organization and the end- user community. The software organization gains added value because high-quality software requires less maintenance effort, fewer bug fixes, and reduced customer support. This enables software engineers to spend more time creating new applications and less on rework. The user community gains added value because the application provides a useful capability in a way that expedites some business process. The end result is (1) greater software product revenue, (2) better profitability when an application supports a business process, and/or (3) improved availability of information that is crucial for the business. 14.2.1 Garvin’s Quality Dimensions David Garvin [Gar87] suggests that quality should be considered by taking a multidi- mensional viewpoint that begins with an assessment of conformance and termi- nates with a transcendental (aesthetic) view. Although Garvin’s eight dimensions of quality were not developed specifically for software, they can be applied when soft- ware quality is considered: Performance quality. Does the software deliver all content, functions, and features that are specified as part of the requirements model in a way that provides value to the end user? Feature quality. Does the software provide features that surprise and delight first-time end users? Reliability. Does the software deliver all features and capability without failure? Is it available when it is needed? Does it deliver functionality that is error-free? Conformance. Does the software conform to local and external software standards that are relevant to the application? Does it conform to de facto design and coding conventions? For example, does the user interface con- form to accepted design rules for menu selection or data input? Durability. Can the software be maintained (changed) or corrected (debugged) without the inadvertent generation of unintended side effects? Will changes cause the error rate or reliability to degrade with time? Serviceability. Can the software be maintained (changed) or corrected (debugged) in an acceptably short time period? Can support staff acquire all information they need to make changes or correct defects? Douglas Adams [Ada93] makes a wry comment that seems appropriate here: “The difference C H A P T E R 1 4 Q U A L I T Y C O N C E P T S 401 pre75977_ch14.qxd 11/27/08 5:51 PM Page 401 between something that can go wrong and something that can’t possibly go wrong is that when something that can’t possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.” Aesthetics. There’s no question that each of us has a different and very subjective vision of what is aesthetic. And yet, most of us would agree that an aesthetic entity has a certain elegance, a unique flow, and an obvious “presence” that are hard to quantify but are evident nonetheless. Aesthetic software has these characteristics. Perception. In some situations, you have a set of prejudices that will influ- ence your perception of quality. For example, if you are introduced to a soft- ware product that was built by a vendor who has produced poor quality in the past, your guard will be raised and your perception of the current soft- ware product quality might be influenced negatively. Similarly, if a vendor has an excellent reputation, you may perceive quality, even when it does not really exist. Garvin’s quality dimensions provide you with a “soft” look at software quality. Many (but not all) of these dimensions can only be considered subjectively. For this reason, you also need a set of “hard” quality factors that can be categorized in two broad groups: (1) factors that can be directly measured (e.g., defects uncovered dur- ing testing) and (2) factors that can be measured only indirectly (e.g., usability or maintainability). In each case measurement must occur. You should compare the software to some datum and arrive at an indication of quality. 14.2.2 McCall’s Quality Factors McCall, Richards, and Walters [McC77] propose a useful categorization of factors that affect software quality. These software quality factors, shown in Figure 14.1, focus on three important aspects of a software product: its operational characteris- tics, its ability to undergo change, and its adaptability to new environments. Referring to the factors noted in Figure 14.1, McCall and his colleagues provide the following descriptions: Correctness. The extent to which a program satisfies its specification and fulfills the customer’s mission objectives. Reliability. The extent to which a program can be expected to perform its intended func- tion with required precision. [It should be noted that other, more complete definitions of reliability have been proposed (see Chapter 25).] Efficiency. The amount of computing resources and code required by a program to perform its function. Integrity. Extent to which access to software or data by unauthorized persons can be controlled. Usability. Effort required to learn, operate, prepare input for, and interpret output of a program. 402 P A R T T H R E E Q U A L I T Y M A N A G E M E N T pre75977_ch14.qxd 11/27/08 5:51 PM Page 402 Maintainability. Effort required to locate and fix an error in a program. [This is a very limited definition.] Flexibility. Effort required to modify an operational program. Testability. Effort required to test a program to ensure that it performs its intended function. Portability. Effort required to transfer the program from one hardware and/or software system environment to another. Reusability. Extent to which a program [or parts of a program] can be reused in other applications—related to the packaging and scope of the functions that the program performs. Interoperability. Effort required to couple one system to another. It is difficult, and in some cases impossible, to develop direct measures2 of these quality factors. In fact, many of the metrics defined by McCall et al. can be measured only indirectly. However, assessing the quality of an application using these factors will provide you with a solid indication of software quality. 14.2.3 ISO 9126 Quality Factors The ISO 9126 standard was developed in an attempt to identify the key quality attributes for computer software. The standard identifies six key quality attributes: Functionality. The degree to which the software satisfies stated needs as indicated by the following subattributes: suitability, accuracy, interoperability, compliance, and security. Reliability. The amount of time that the software is available for use as indi- cated by the following subattributes: maturity, fault tolerance, recoverability. C H A P T E R 1 4 Q U A L I T Y C O N C E P T S 403 PRODUCT OPERATION PRODUCT TRANSITIONPRODUCT REVISION Correctness Usability Efficiency Reliability Integrity Maintainability Flexibility Testability Portability Reusability Interoperability FIGURE 14.1 McCall’s software quality factors uote: “The bitterness of poor quality remains long after the sweetness of meeting the schedule has been forgotten.” Karl Weigers (unattributed quote) 2 A direct measure implies that there is a single countable value that provides a direct indication of the attribute being examined. For example, the “size” of a program can be measured directly by counting the number of lines of code. pre75977_ch14.qxd 11/27/08 5:51 PM Page 403 Usability. The degree to which the software is easy to use as indicated by the following subattributes: understandability, learnability, operability. Efficiency. The degree to which the software makes optimal use of system resources as indicated by the following subattributes: time behavior, resource behavior. Maintainability. The ease with which repair may be made to the software as indicated by the following subattributes: analyzability, changeability, stability, testability. Portability. The ease with which the software can be transposed from one environment to another as indicated by the following subattributes: adapt- ability, installability, conformance, replaceability. Like other software quality factors discussed in the preceding subsections, the ISO 9126 factors do not necessarily lend themselves to direct measurement. However, they do provide a worthwhile basis for indirect measures and an excellent checklist for assessing the quality of a system. 14.2.4 Targeted Quality Factors The quality dimensions and factors presented in Sections 14.2.1 and 14.2.2 focus on the software as a whole and can be used as a generic indication of the quality of an application. A software team can develop a set of quality characteristics and associ- ated questions that would probe3 the degree to which each factor has been satisfied. For example, McCall identifies usability as an important quality factor. If you were asked to review a user interface and assess its usability, how would you proceed? You might start with the subattributes suggested by McCall—understandability, learnability, and operability—but what do these mean in a pragmatic sense? To conduct your assessment, you’ll need to address specific, measurable (or at least, recognizable) attributes of the interface. For example [Bro03]: Intuitiveness. The degree to which the interface follows expected usage patterns so that even a novice can use it without significant training. • Is the interface layout conducive to easy understanding? • Are interface operations easy to locate and initiate? • Does the interface use a recognizable metaphor? • Is input specified to economize key strokes or mouse clicks? • Does the interface follow the three golden rules? (Chapter 11) • Do aesthetics aid in understanding and usage? 404 P A R T T H R E E Q U A L I T Y M A N A G E M E N T uote: “Any activity becomes creative when the doer cares about doing it right, or better.” John Updike Although it’s tempting to develop quantitative measures for the quality factors noted here, you can also create a simple checklist of attributes that provide a solid indication that the factor is present. 3 These characteristics and questions would be addressed as part of a software review (Chapter 15). pre75977_ch14.qxd 11/27/08 5:51 PM Page 404 Efficiency. The degree to which operations and information can be located or initiated. • Does the interface layout and style allow a user to locate operations and information efficiently? • Can a sequence of operations (or data input) be performed with an economy of motion? • Are output data or content presented so that it is understood immediately? • Have hierarchical operations been organized in a way that minimizes the depth to which a user must navigate to get something done? Robustness. The degree to which the software handles bad input data or inap- propriate user interaction. • Will the software recognize the error if data at or just outside prescribed boundaries is input? More importantly, will the software continue to operate without failure or degradation? • Will the interface recognize common cognitive or manipulative mistakes and explicitly guide the user back on the right track? • Does the interface provide useful diagnosis and guidance when an error condition (associated with software functionality) is uncovered? Richness. The degree to which the interface provides a rich feature set. • Can the interface be customized to the specific needs of a user? • Does the interface provide a macro capability that enables a user to identify a sequence of common operations with a single action or command? As the interface design is developed, the software team would review the design prototype and ask the questions noted. If the answer to most of these questions is “yes,” it is likely that the user interface exhibits high quality. A collection of questions similar to these would be developed for each quality factor to be assessed. 14.2.5 The Transition to a Quantitative View In the preceding subsections, I have presented a variety of qualitative factors for the “measurement” of software quality. The software engineering community strives to develop precise measures for software quality and is sometimes frustrated by the subjective nature of the activity. Cavano and McCall [Cav78] discuss this situation: The determination of quality is a key factor in every day events—wine tasting contests, sporting events [e.g., gymnastics], talent contests, etc. In these situations, quality is judged in the most fundamental and direct manner: side by side comparison of objects under identical conditions and with predetermined concepts. The wine may be judged according to clarity, color, bouquet, taste, etc. However, this type of judgment is very sub- jective; to have any value at all, it must be made by an expert. C H A P T E R 1 4 Q U A L I T Y C O N C E P T S 405 pre75977_ch14.qxd 11/27/08 5:51 PM Page 405 Subjectivity and specialization also apply to determining software quality. To help solve this problem, a more precise definition of software quality is needed as well as a way to derive quantitative measurements of software quality for objective analysis. . . . Since there is no such thing as absolute knowledge, one should not expect to measure software quality exactly, for every measurement is partially imperfect. Jacob Bronkowski described this paradox of knowledge in this way: “Year by year we devise more precise instruments with which to observe nature with more fineness. And when we look at the observations we are discomfited to see that they are still fuzzy, and we feel that they are as uncertain as ever.” In Chapter 23, I’ll present a set of software metrics that can be applied to the quan- titative assessment of software quality. In all cases, the metrics represent indirect measures; that is, we never really measure quality but rather some manifestation of quality. The complicating factor is the precise relationship between the variable that is measured and the quality of software. 1 4 . 3 T H E S O F T WA R E Q U A L I T Y D I L E M M A In an interview [Ven03] published on the Web, Bertrand Meyer discusses what I call the quality dilemma: If you produce a software system that has terrible quality, you lose because no one will want to buy it. If on the other hand you spend infinite time, extremely large effort, and huge sums of money to build the absolutely perfect piece of software, then it’s going to take so long to complete and it will be so expensive to produce that you’ll be out of busi- ness anyway. Either you missed the market window, or you simply exhausted all your resources. So people in industry try to get to that magical middle ground where the prod- uct is good enough not to be rejected right away, such as during evaluation, but also not the object of so much perfectionism and so much work that it would take too long or cost too much to complete. It’s fine to state that software engineers should strive to produce high-quality systems. It’s even better to apply good practices in your attempt to do so. But the situation discussed by Meyer is real life and represents a dilemma for even the best software engineering organizations. 14.3.1 “Good Enough” Software Stated bluntly, if we are to accept the argument made by Meyer, is it acceptable to produce “good enough” software? The answer to this question must be “yes,” because major software companies do it every day. They create software with known bugs and deliver it to a broad population of end users. They recognize that some of the functions and features delivered in Version 1.0 may not be of the highest quality and plan for improvements in Version 2.0. They do this knowing that some cus- tomers will complain, but they recognize that time-to-market may trump better qual- ity as long as the delivered product is “good enough.” 406 P A R T T H R E E Q U A L I T Y M A N A G E M E N T When you’re faced with the quality dilemma (and everyone is faced with it at one time or another), try to achieve balance— enough effort to produce acceptable quality without burying the project. pre75977_ch14.qxd 11/27/08 5:51 PM Page 406 Exactly what is “good enough”? Good enough software delivers high-quality func- tions and features that users desire, but at the same time it delivers other more obscure or specialized functions and features that contain known bugs. The soft- ware vendor hopes that the vast majority of end users will overlook the bugs because they are so happy with other application functionality. This idea may resonate with many readers. If you’re one of them, I can only ask you to consider some of the arguments against “good enough.” It is true that “good enough” may work in some application domains and for a few major software companies. After all, if a company has a large marketing budget and can convince enough people to buy version 1.0, it has succeeded in locking them in. As I noted earlier, it can argue that it will improve quality in subsequent versions. By delivering a good enough version 1.0, it has cornered the market. If you work for a small company be wary of this philosophy. When you deliver a good enough (buggy) product, you risk permanent damage to your company’s repu- tation. You may never get a chance to deliver version 2.0 because bad buzz may cause your sales to plummet and your company to fold. If you work in certain application domains (e.g., real-time embedded software) or build application software that is integrated with hardware (e.g., automotive soft- ware, telecommunications software), delivering software with known bugs can be negligent and open your company to expensive litigation. In some cases, it can even be criminal. No one wants good enough aircraft avionics software! So, proceed with caution if you believe that “good enough” is a short cut that can solve your software quality problems. It can work, but only for a few and only in a limited set of application domains.4 14.3.2 The Cost of Quality The argument goes something like this—we know that quality is important, but it costs us time and money—too much time and money to get the level of software quality we really want. On its face, this argument seems reasonable (see Meyer’s comments ear- lier in this section). There is no question that quality has a cost, but lack of quality also has a cost—not only to end users who must live with buggy software, but also to the software organization that has built and must maintain it. The real question is this: which cost should we be worried about? To answer this question, you must understand both the cost of achieving quality and the cost of low-quality software. The cost of quality includes all costs incurred in the pursuit of quality or in per- forming quality-related activities and the downstream costs of lack of quality. To understand these …