Vondran Andre's Posts - Knol Stuff2024-03-29T13:12:39ZVondran Andrehttps://knolstuff.com/profile/VondranAndrehttps://storage.ning.com/topology/rest/1.0/file/get/1953550411?profile=RESIZE_48X48&width=48&height=48&crop=1%3A1https://knolstuff.com/profiles/blog/feed?user=1ixu5xrzgmqja&xn_auth=noTest Plans By André Vondrantag:knolstuff.com,2008-01-21:1781665:BlogPost:356462008-01-21T19:09:22.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: #006666" xml:lang="EN-US">Problem…</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: #006666" xml:lang="EN-US">Problem</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: black" xml:lang="EN-US"></span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">A common pitfall of testing activity is that it doesn’t test the right functionality, or it spends too much time in some areas, or is too detailed. These are the symptoms of poor planning, and can be mitigated by the production of a test plan.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: #006666" xml:lang="EN-US">Solution</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">Before any testing can commence, it is first important to understand the scope of:</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">What is being tested:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Understand functionality, interfaces and architecture in and out of scope</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">How it is to be tested:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Understand the range of test styles (boundary, non-functional, user experience) that is necessary</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">What are the constraints:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Understand time and resource that are available for testing</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">What is the fall-back</span></b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">: What are the consequences of undetermined bugs? Not every application is life-critical, and often defects can be managed as part of an iterative cycle of development.</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">Depending on the answers to the above a completely different style of testing may be appropriate.</font></span></p>'Life is too short for manual testing' By André Vondrantag:knolstuff.com,2008-01-21:1781665:BlogPost:356442008-01-21T19:08:23.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN" style="mso-ansi-language: EN" xml:lang="EN"><font face="Times New Roman">My experience as a tester shows that the nastiest bugs are often discovered by manual testing. But when you do discover them manually, the best…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN" style="mso-ansi-language: EN" xml:lang="EN"><font face="Times New Roman"></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN" style="mso-ansi-language: EN" xml:lang="EN"><font face="Times New Roman">My experience as a tester shows that the nastiest bugs are often discovered by manual testing. But when you do discover them manually, the best strategy is to write automated tests for them, so that you'll check your application in that particular area from that moment on, via an automated test suite which runs in your continuous integration system.<br/><br/>You do have an automated test suite, right? And it does run periodically (daily or upon on every check-in) in a continuous integration system, right? And you have everything set up so that you're notified by email or RSS feeds when something fails, right? And you fix failures quickly so that everything turns back to green, because you know that too much red, too often, leads to</font> <a href="http://www.artima.com/intv/fixitP.html"><span style="FONT-FAMILY: Verdana"><font color="#6699CC">broken windows</font></span></a> <font face="Times New Roman">and bit rot, right?<br/><br/>If you answered No to any of these questions, then you are not testing your application, period (but you already knew this if you took our tutorial -- it was on the last slide :-)</font></span></p>Four implementation styles for workflow tests By André Vondrantag:knolstuff.com,2008-01-21:1781665:BlogPost:356422008-01-21T19:07:17.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Many of the tests that currently use browser-driving tools like</font> <a href="http://www.openqa.org/selenium/"><font color="#0000FF" face="Times New Roman">Selenium</font></a><font face="Times New Roman">,…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 8pt" xml:lang="EN-US"> </span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Many of the tests that currently use browser-driving tools like</font> <a href="http://www.openqa.org/selenium/"><font face="Times New Roman" color="#0000FF">Selenium</font></a><font face="Times New Roman">,</font> <a href="http://www.openqa.org/watir/"><font face="Times New Roman" color="#0000FF">Watir</font></a><font face="Times New Roman">, or Silk could use different implementation technologies. In the rest of this note, I describe the pros and cons of browser driving, HTTP driving, the Rails variant of HTTP driving, and app-layer driving.<br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><strong><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Context</font></span></strong></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">There are certain kinds of automated <b style="mso-bidi-font-weight: normal">business-facing</b> tests whose job is to show how a user can accomplish some task. I’m going to call them “workflow tests.” They usually involve (1) navigation through the application and (2) judicious checking that the application is where it should be with at least some of the right data visible. (I mention “judicious” because I believe demonstrations that a page contains every bit of what it should contain ought to be done by other tests and not repeated in these. Stripping the checking down will increase maintainability at a most-likely-negligible risk.)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">The purposes of these tests are:</font></span></p>
<ol style="MARGIN-TOP: 0cm" type="1">
<li class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">to help the programmer understand what the business wants.</font></span></li>
<li class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">to help everyone agree what it means to say the current story is done.</font></span></li>
<li class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">to help the product director (or perhaps a user experience designer) to think through what it is that she wants—to be a concrete way of exploring ideas.</font></span></li>
<li class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">to find bugs introduced by programmers making changes. However: if these tests catch many “regression” bugs, I think you’re doing something wrong. Most should be caught earlier, by other tests. Fix those tests, or fix the infrastructure for your app to make more kinds of bugs impossible or unlikely.</font></span></li>
</ol>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Here’s an example of such a test:</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">to show emptying cart after returning to session:</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">user visits site</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US"> (user is on book catalog page)</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">User adds "Everyday Scripting"</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">User abandons site for 2 days</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">User visits site</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">(user has in his cart "Everyday Scripting"</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">User empties cart</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Monotype Corsiva'" xml:lang="EN-US">(user has nothing in his cart)</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Except for usual spacing and indentation, it looks pretty much like English. (One thing that may not be clear is that the parenthetical comments are what the test checks.) A natural language is appropriate, given a workflow test’s audience. But, because computers are so blasted picky, you have to clutter the test up a bit.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">It doesn’t take much practice for a product director to ignore the clutter.</font></span></p>
<p><font face="Times New Roman"><i><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">In each case below, the test continues to look the same.</span></i> <span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">That means someone has to write a <i>translation layer</i> that converts the language of the test to some pickier, more detailed, more techie language. The pros and cons of each approach are the pros and cons of its characteristic translation layer.</span></font></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">(Note: With tools like Watir and Selenium, you could write the tests in the language of browser actions—click and the like—but that hampers all but the least important goal of workflow tests. Since it’s also a maintainability nightmare, I’m going to ignore that approach. Even with button-pressing tools, you hide the button pressing behind the translation layer.)</font></span></p>
<p><font face="Times New Roman"><strong><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">Driving the browser</span></strong> </font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">In the first approach, the translation layer need know nothing about the app other than that it’s driven through a browser. There’s a server out there somewhere, but no one cares.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">This style makes it <b>easiest to learn how to write the translation layer</b>. Both what you’re translating from (English) and to (buttons, text fields) are familiar to most anyone involved in computers. As a result, you can <b>start testing quickly</b>. The tests are satisfying because they <b>test what the user sees</b>. The app is used exactly as a real user would use it.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">There are difficulties, though. Because the user interface can change a lot, the translation layer tends to be <b>a maintainability problem</b>. You can also run into <b>technical roadblocks</b>. One week, I was at a site that was blocked trying to figure out how to make Watir deal with certificate popups in Internet explorer. Two weeks later, the problem was making Selenium deal with IE’s file-upload popups. (That time, we moved to a different technology.)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Finally, this style <b>doesn’t mesh smoothly with test-driven design</b>. At least it doesn’t feel that way to me—I could be wrong&mdash. The problem is you have to decide fiddly little details of the interface (”what’s the name of the submit button?”) before you can even see the test fail.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Driving HTTP</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Tools like</font> <a href="http://httpunit.sourceforge.net/"><font face="Times New Roman" color="#0000FF">HTTPUnit</font></a> <font face="Times New Roman">and</font> <a href="http://webtest.canoo.com/webtest/manual/WebTestHome.html"><font face="Times New Roman" color="#0000FF">Canoo WebTest</font></a> <font face="Times New Roman">take a different approach. They know that the browser has to speak to the application server via the HTTP protocol. Instead of driving the browser, they produce and consume HTTP, driving the app directly. In effect, they pretend to be the browser.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Their translation layer is <b>likely more maintainable</b>. It doesn’t have to fiddle with a bunch of typing and button presses—it merely packages up the data into a relatively simple structure and sends it off. Because these tests work below the browser, they can <b>avoid the technical complications browsers bring</b> (like the popup problems I mentioned above).</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">On the other hand, because tests don’t involve the browser, they <b>can’t check everything that the browser does</b>. Even if the tool you’re scripting handles Javascript (as HTTPUnit says it does), you don’t know if its Javascript has the same idiosyncrasies as Internet Explorer’s does. If poking at Javascript is part of what your judicious set of workflow checks, <b>you might have to fall back on manual work for some tests or trust that earlier tests suffice</b>. (For example, you might have tested your Javascript in isolation with</font> <a href="http://www.jsunit.net/"><font face="Times New Roman" color="#0000FF">JSUnit</font></a><font face="Times New Roman">, or you may rely on well-tested javascript-generating components like</font> <a href="http://www.rubyonrails.org/"><font face="Times New Roman" color="#0000FF">Rails</font></a> <font face="Times New Roman">or the</font> <a href="http://code.google.com/webtoolkit/"><font face="Times New Roman" color="#0000FF">Google Web Toolkit</font></a><font face="Times New Roman">.)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">I’ve found that people—technical and nontechnical alike—have an almost instinctive aversion to not “testing what the user sees.” I think that reaction is overblown, but it’s nevertheless a real thing you need to cope with if you go this route.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Finally, this approach is <b>harder to learn</b> than the previous one, and I think the <b>scripting skills need to be stronger</b>.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><font face="Times New Roman"><b><span lang="EN-US" xml:lang="EN-US">The Rails variant on driving HTTP</span></b> </font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Rails extends the previous idea by linking the test code into the main body of the application. That is, rather than construct an HTTP Request that flies across some network, is received by the app, funneled through some application-wide Request-handling code, and then delivered to the code that handles that particular request, a <i>Rails integration test</i> skips the network. From inside the app, it calls the application-wide Request-handling code directly.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">The main additional advantage to this approach is that <b>tests can make use of application support code</b>. For example, a test that wants to check whether an order has <i>really</i> been saved to the database doesn’t have to open a database connection and SQL around through it; it can use the Rails object-relational-database mapping code (which is so nice and easy that it will make you cry).</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Easier observation means <b>fewer distorted tests</b>. Sometimes a step in a workflow causes something to happen in the application, but that something isn’t immediately visible. Without direct access to support code, these tests often have to go through some convoluted sequence of steps to let the test either see or infer that the right thing happened.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">I also expect—although I do not know from personal experience—that this integration will, in the end, considerably <b>reduce the total effort of making the translation layer</b>.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">The first difficulty of this approach is that <b>someone must have built the internal test-support framework already</b>. Often, there isn’t one. It may not even be possible for a test to use the application-wide Request-handling code without jumping through a ridiculous number of hoops. And things are not necessarily trivial even if the support code exists: of all the approaches, this one probably <b>requires the translation layer writer to have the greatest scripting skill</b> and that she <b>learn the most about the application’s internals</b>. Moreover, these tests lead to the same (perhaps slightly greater) doubts about what goes untested as do those of the previous approach, plus one new one: <b>How much should you trust the application to tell the truth about its internals?</b> (Rails says it saved that Order to the database. Why should you believe it?)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Driving the Application layer</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">In many systems, especially well-designed ones, there’s a <b style="mso-bidi-font-weight: normal">close correspondence</b> between the classes and methods of the code and the nouns and verbs of the business domain. In such systems, the user interface is a way for the user to express clear and simple ideas in a welter of visually-appealing detail. Then the code behind the user interface translates all that detail back into clear and simple classes and methods.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Huh. What if we could go below the user interface altogether?</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Many applications have an <b style="mso-bidi-font-weight: normal">application controller</b> or workflow layer that orchestrates user tasks. Roughly, a single HTTP request might translate directly into a call to that layer, which then describes (in a fairly abstract way) what the user can do next to what business objects. That description drives the creation of the next screen. You send business language in, get business language out.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Given that our tests are written in business language, having them call the application layer directly makes a lot of sense. (These tests could run within the application, in the Rails style, or they could communicate with the application layer from outside through some web-services-like mechanism.) That probably <b>requires the smallest test translation layer, and one that’s reasonably straightforward to write</b>. Further, because the language of the business is relatively slow to change, <b>these tests will likely break less often than tests that work with some part of the UI layer</b>. Finally, <b>this style meshes well with test-driven design</b>: when the tests come first, they help decide upon the language of the application layer and thus help <b>tune the application to the business</b>.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">However, your application either has such a layer or it doesn’t. If it doesn’t, adding it <b>might require a heavy programmer investment</b>. Moreover, <b>that investment is probably not reusable</b>. This isn’t a tool like Watir or HTTPUnit, which can drive any web app, or even the Rails integration test framework, which can be used on any Rails app. Here, everything’s written to one application layer. Moreover, this approach <b>leaves the most untested</b>. Not only are you not testing Javascript, you’re not testing the receipt and processing of HTTP messages or even whether anything like the right HTML is produced. That has to be tested separately.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Summary</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">All of these approaches—or a combination—can work just fine. I think the most important thing is to keep the number of these tests small. Don’t use Selenium to check any property of the app that can be checked in programmer tests. Don’t automate what’s better tested as a part of manual exploratory testing.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">That is, despite the Rails framework being way cool, I still like testing against an app layer better.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>QA( both: PQA & SQA), QC, Software Testing By André Vondrantag:knolstuff.com,2008-01-21:1781665:BlogPost:356402008-01-21T18:57:08.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: #006666" xml:lang="EN-US">Problem…</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: #006666" xml:lang="EN-US">Problem</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: black" xml:lang="EN-US"></span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">Testing is not an activity that can be made up as you go along. By definition success targets must be set in advance. This is typically be done be defining a test script: a repeatable set of instructions that when followed determine the success or otherwise of some application.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">While script writing must be done in advance this can often be difficult to do, particularly in cases where the level of system documentation is poor. It is thus essential to understand what is appropriate to go into the script.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"><font face="Times New Roman"><b><span lang="EN-US" style="FONT-SIZE: 18pt; COLOR: #006666" xml:lang="EN-US">Solution</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto; mso-outline-level: 2"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">The challenge is defining scripts that meet the following objectives.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Keep agile:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Often testing out of proportion of what needs to be tested</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Ensure complete coverage:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">It is important that there are no significant holes in the testing</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Develop in cooperation of IT:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Develop scripts in close cooperation with the developers – testers often feel the preparation activity must be done in a vacuum as they fear that involvement of other parties may contaminate the impartiality – this is a mistake – typically time is very constrained at this point of the project so any ways of right sizing are of benefit.</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Ensure separation of testers to IT:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">The area where you want to keep clear separation in terms of responsibility is the execution of the tests (rather than the formulating of test scripts). If the scripts are clear and concise enough to allow a 3<sup>rd</sup> party without any domain or functional experience to run, then you have succeeded in pitching the scripts at the right level.</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Define the evidence basis:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">How is success verified? SOX mandates that testing evidence is to be recorded in certain cases.</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Ensure the tests are right for what is being tested:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Unless the application is very large, or has very complex STP/ algorithms the following areas should are really all that should be considered for the typical web-based application:</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Quality assurance:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Given that the major part the development is user-facing, <i><u>quality assurance</u></i> suite of scripts are to confirm that the user experience is operating in a consistent way (for example ensuring that basic standards in UI design are adhered to such as formatting, accessibility, error-handling, alignment and spelling). Support for different browsers and operating systems is also to be tested. The functionality of security (availability to functions for those with/without a valid account/password) is also covered. The format of these scripts is a checklist against all screens.</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">End-to-end:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">The majority of screens form part of an end-to-end enterprise, so it is important to demonstrate that the users can navigate through the screens to achieve their end-goal, without any logical breaks or unexpected behaviour. It is also important to demonstrate that data input in an earlier stage is carried forward in a consistent manner. <i><u>User-experience</u></i> suite of tests therefore briefly describes scenarios involving a number of screens to ensure all stages and the end result occur as expected.</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Boundary testing:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Testing of the behaviour of both inputs and outputs is shown in the <i><u>boundary-test</u></i> suite of scripts. With the inputs this works by examining that values of each control in isolation and stressing them with boundary conditions or inconsistent values to their type, for outputs the boundary values of what can be accepted by the receiving component are examined, and tests are contrived to see if these can be exceeded (i.e. it drives us to ensure that test coverage is complete, but we are not actually testing the interface directly).</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Interface testing:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">This is a technical test. Checks to ensure input/output to interface are as expected, and that unavailability of interface does not cause unexpected/unsupported application behaviour in both synchronous and asynchronous modes</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">Load/SLA testing:</span></b> <span lang="EN-US" style="COLOR: black" xml:lang="EN-US">For purposes of this agile testing these can be often be combined. Load testing will attempt to see the upper number of users the application can support.</span></font></p>can mentor you on the adoption of agile testing methodologies By André Vondrantag:knolstuff.com,2008-01-21:1781665:BlogPost:356382008-01-21T18:55:05.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">What is agile testing? Put simply it is the close collaboration between the test writer and the developers to ensure test scripts can both be rapidly created and are robust.…</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><b><span lang="EN-US" style="COLOR: black" xml:lang="EN-US">What is agile testing? Put simply it is the close collaboration between the test writer and the developers to ensure test scripts can both be rapidly created and are robust.</span></b></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><font face="Times New Roman"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"></span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"><span lang="EN-US" style="COLOR: black" xml:lang="EN-US"><font face="Times New Roman">I can mentor you on the adoption of agile testing methodologies, both industries standard or bespoke. Software engineering can often be complex. To succeed, it demands the use of IT methodologies. However off-the-shelf methodologies can often be unwieldy or inflexible.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-margin-top-alt: auto; mso-margin-bottom-alt: auto"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><a href="http://www.codel-services.com/Analysis/ent_intro.htm"></a><font face="Times New Roman">Test Plans</font></span></b> <span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">are designed to set the scope of any testing initiative. Often testing activity fails as the scope of what is being tested and how it is to be tested is poorly understood.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><a href="http://www.codel-services.com/Analysis/FwdEngMethod.htm"></a><font face="Times New Roman">Test Scripts</font></span></b> <span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">are designed to be an agile solution to testing. They are formulated specifically for web-based applications and focus on what is really required to ensure that such applications can be proved to meet client expectations. Once completed, they can be given to a third party without any specific knowledge of the application.</font></span></p>Agile testing By André Vondrantag:knolstuff.com,2008-01-21:1781665:BlogPost:356282008-01-21T17:26:56.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Q1: Why use Agile methods?</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><br></br>Because they work. If they don't work in your organization, there's no reason whatsoever to use them—certainly not so you can brag about being "agile."…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Q1: Why use Agile methods?</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><br/>Because they work. If they don't work in your organization, there's no reason whatsoever to use them—certainly not so you can brag about being "agile."</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">When you do make them work, they make projects more manageable. You may or may not get quicker project completion.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">You may or may not save money. But you will get better customer satisfaction, and you will stay on top of your projects and be able to predict how much they will cost and how long they will take.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">You may or may not stay out of trouble, but if trouble comes, you'll see it coming in time to do something about it. That's what project management is all about.<br/><br/><b>Q2: Biggest challenge of implementing Agile methods?</b></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><br/>People tend to prefer familiarity to comfort. It's unfamiliar, so your challenge is to manage the human reactions to change. For example, instead of pushing every team to be "agile," I like to have the management say, "Agile methods are not for everyone.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">We want to start with the best qualified teams, so if you want your team to be chosen for our first trial, you will have to make a presentation to management about why you are best qualified." (In other words, what is your "story.")</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">This approach has worked well, so well, in fact, that in one case in Austria, the programmers' union came in and insisted that everyone be given an equal chance to try these new methods.<br/><br/>I have written extensively about change artistry in the fourth volume of my Quality Software Management series, "Anticipating Change."<br/><br/><b>Q3: In what environment will Agile be most successful?</b></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><br/>The change to agile methods will be most successful in those organizations with an agile management approach to converting to agile methods.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Unfortunately, I've observed a number of organizations where agile methods are introduced like a waterfall project--a huge up-front planning effort, then an attempt to convert an entire organization at one fell swoop.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">To be successful, the conversion has to be in small increments, with corrections made at each increment.<br/><br/></font><font face="Times New Roman"><b>Q4: What is the future of Agile?</b></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><b><br/></b>First we will drop the capital A. Then we will drop the term "agile" altogether. Agile methods will be successful if and when we stop seeing them as anything other than normal, sensible, professional methods of developing software.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">agile will be the next standard, when people realize that CMMI is obsolete</font></span></b></p>
<p style="TEXT-ALIGN: left"><img style="WIDTH: 92px; HEIGHT: 79px" height="216" alt="" src="http://storage.ning.com/topology/rest/1.0/file/get/1974645244?profile=RESIZE_320x320" width="300"/></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-ALIGN: center" align="center"><b><span style="FONT-SIZE: 18pt; FONT-FAMILY: 'Monotype Corsiva'; mso-ansi-language: ES">André Vondran</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-ALIGN: center" align="center"><b><span style="mso-ansi-language: ES"><font face="Times New Roman"><font size="3">PQA & SQA Analyst</font></font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-ALIGN: center" align="center"><font size="3"><b><span style="FONT-FAMILY: 'Franklin Gothic Medium'; mso-ansi-language: ES"><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></span></b></font></p>Project Quality Assurance By André Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342212008-01-17T14:52:17.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><strong><u>Acceptance Tests</u></strong></span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Write the acceptance tests (meaning any tests needed beyond unit and integration tests: system, functional, end-to-end, performance, security, whatever) during the planning phases of the project, rather than after…</span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><strong><u>Acceptance Tests</u></strong></span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Write the acceptance tests (meaning any tests needed beyond unit and integration tests: system, functional, end-to-end, performance, security, whatever) during the planning phases of the project, rather than after development is complete. <br/>More importantly, if you’re trying an agile approach, write executable acceptance tests.<br/>Executable acceptance tests serve two purposes.</span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">First of all, they’re a mechanism to involve your ‘customer’ or business expert in defining the tests which will verify that the level of quality and functionality he has specified is met. <br/>Secondly, they are a part of your automated acceptance test framework. <br/>If you can, avoid having to re-work your acceptance test cases into input to test automation tools. There are quite a few test tool frameworks which allow this. <br/>The concept of data-driven testing isn’t new to XP, and there are plenty of tools which let you define tests in an Excel spreadsheet and then use these as input to your automated tests.</span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><strong><u>XP-style test frameworks abound, as well.</u></strong> <br/>It doesn’t matter what tool you’re going to use. <br/>Defining acceptance test cases up front helps your programmers write the correct code. <br/>It gives you a huge jump on your testing too.<br/>A word about test plans. Nobody ever reads them, so try not to write them. <br/>They’re cumbersome to write and impossible to maintain. Just write test cases.</span></i> <i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><br/>If your organization requires you to deliver a test plan, try to include information that will be useful to you, such as a release plan, and refer to the test cases as much as possible.</span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><u><strong>Technical Design</strong></u></span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Depending on your technical skills, technical design meetings may be painful for you. Try to attend them anyway, if you have the opportunity. <br/>If you can even ask one question or make one comment that leads to a more testable design, you’re way ahead of the game.<br/>XP has the wonderful benefit of making everyone on the development team conscious of testability. <br/>If you have to automate and perform acceptance tests, and it’s hard to do because your GUI is thick or hooked up to the back end in a way that you can’t test much behind the GUI, your next application is bound to be designed to be more testable. <br/>Unfortunately, your traditional development project won’t give you this golden opportunity. <br/>You can still raise these issues yourself (tactfully, of course) at design meetings. <br/>You can also continue to help facilitate communication between the business stakeholders and the programmers. <br/>You understand the application pretty well by now, and you probably are more familiar with the technical issues than the folks on the business side.</span></i></p>
<p><i><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><strong><u>Remember that communication value!</u></strong></span></i></p>Testing the Documentation by André Vondran - This is a MUSTtag:knolstuff.com,2008-01-17:1781665:BlogPost:342192008-01-17T14:46:28.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p>Whether or not you attend the meetings, you can add value to the project by ‘<em>testing</em>’ the documentation. <br></br>This isn’t an XP concept, but it is linked to flushing out hidden assumptions during XP planning games. <br></br>One way to test documents is to read through the document, writing a list of assertions. <br></br>For example, maybe you’re reading requirements for an order entry system and have come up with these assertions:</p>
<ul>
<li>"The system will capture the customer service…</li>
</ul>
<p>Whether or not you attend the meetings, you can add value to the project by ‘<em>testing</em>’ the documentation. <br/>This isn’t an XP concept, but it is linked to flushing out hidden assumptions during XP planning games. <br/>One way to test documents is to read through the document, writing a list of assertions. <br/>For example, maybe you’re reading requirements for an order entry system and have come up with these assertions:</p>
<ul>
<li>"The system will capture the customer service agent’s name".</li>
<li>"The new system will be 50% faster than the old system".</li>
<li>"The billing address will be required."</li>
<li>"The zip code will automatically populate the city name".</li>
</ul>
<p>Go over your assertions with the document’s author and clarify any ambiguous or contradictory assertions.</p>
<p><br/>Will the system capture the agent’s name by means of the agent logging into the system? <br/>What exactly in the new system will be 50% faster – the response time on each page? <br/>The time for placing an order? Do we have a baseline for the old system? <br/>If not, how do we quantify this requirement so we know it’s met? <br/>What about cases where multiple cities share the same zip code? <br/>Are there any optional fields in the address (such as the second address line)? <br/>Clearing up these issues now will save time for the programmers later, just as it does during an XP planning game.</p>
<p>If you’ve got a big project where the team sees a problem with finishing on time, see if there is a way you can suggest an XP approach to the planning.</p>
<p><br/>If the requirements can be grouped into story-like chunks, estimated by the programmers, and prioritized by the stakeholders, the team can take an iterative approach which will help ensure that some percentage of the necessary features are 100% ready by the deadline, even if less critical features are not yet implemented.</p>
<p><br/><strong>If you’re new to agile processes or new to this team, it might not be the right time to take such a bold step, but keep it in mind.</strong></p>
<p style="TEXT-ALIGN: left"><img style="WIDTH: 55px; HEIGHT: 74px" height="437" alt="" src="http://storage.ning.com/topology/rest/1.0/file/get/1974644941?profile=RESIZE_320x320" width="300"/></p>
<p><em>André Vondran</em></p>
<p><strong>SQA Engineer</strong></p>Project Planning by André Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342162008-01-17T14:40:12.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">Most projects begin with some kind of requirements gathering. The business experts may write business requirements.</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">This is often followed by the development team producing…</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">Most projects begin with some kind of requirements gathering. The business experts may write business requirements.</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">This is often followed by the development team producing functional specifications and/or technical specifications.</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">If you’re working on a project which is producing documents like this, try to get in on the planning meetings.</font></span></p>
<p/><p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">I know, the last thing you need in your life is more meetings. Requirements gathering meetings can be a great opportunity to put your agile skills to work. You’ll get a head start in understanding the project.</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">By alternately looking at the requirements from the point of view of the end users, business experts and programmers, you can ask questions that flush out hidden assumptions.</font></span></p>André Vondran: SQA Valuestag:knolstuff.com,2008-01-17:1781665:BlogPost:342142008-01-17T14:37:43.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p></p>
<p>Earlier in this article I referred to the values by which XP teams develop software.</p>
<ul>
<li>Communication</li>
<li>Simplicity</li>
<li>Feedback</li>
<li>Courage</li>
</ul>
<p>Some people have added their own values to this list – the one I liked best is ‘<em>enjoyment</em>’. <br></br>Extreme Programming is the only ‘official’ agile method I have actually practiced, but from what I have learned of agile software development in general, you can always turn to these values for…</p>
<p/><p>Earlier in this article I referred to the values by which XP teams develop software.</p>
<ul>
<li>Communication</li>
<li>Simplicity</li>
<li>Feedback</li>
<li>Courage</li>
</ul>
<p>Some people have added their own values to this list – the one I liked best is ‘<em>enjoyment</em>’. <br/>Extreme Programming is the only ‘official’ agile method I have actually practiced, but from what I have learned of agile software development in general, you can always turn to these values for guidance.</p>
<p><br/><strong>Communication is usually your first job.</strong></p>
<p><br/>Whether you’re new to a position or you just want to try something new, you’re going to have to talk to your coworkers and your customers. <strong>Most importantly, you need to listen. <br/></strong>As you proceed in your agile journey, remember the XP mantra: "Do the simplest thing that could possibly work".</p>
<p/><p><strong>As a tester, feedback is your core function</strong>. You’ll need plenty of courage to try agile practices in a less-than-agile environment.</p>
<p><br/>Let’s explore a typical project life cycle and see how XP practices can help you with testing, whether or not your development team is taking advantage of them.</p>
<p/><p>André Vondran</p>
<p>SQA Engineer</p>
<p style="TEXT-ALIGN: left"><img style="WIDTH: 47px; HEIGHT: 58px" height="421" alt="" src="http://storage.ning.com/topology/rest/1.0/file/get/1974645357?profile=RESIZE_320x320" width="300"/></p>
<p><br/></p>XP Testing Without an XP Team, By André Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342082008-01-17T14:32:30.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p></p>
<p>After my rewarding, if challenging, experience as a tester on XP teams, the bad economic times caught up with me and I had to take a job back on the "<strong>dark side</strong>".</p>
<p>Oh, my new company was great and they were interested in XP, but they were a tiny shop with a huge job to do and it felt to me like barely controlled chaos.</p>
<p>My initial attempts to apply XP practices, such as having daily XP-style stand-ups with my test team, were disasters. I just had to hunker…</p>
<p/><p>After my rewarding, if challenging, experience as a tester on XP teams, the bad economic times caught up with me and I had to take a job back on the "<strong>dark side</strong>".</p>
<p>Oh, my new company was great and they were interested in XP, but they were a tiny shop with a huge job to do and it felt to me like barely controlled chaos.</p>
<p>My initial attempts to apply XP practices, such as having daily XP-style stand-ups with my test team, were disasters. I just had to hunker down, hire people that I thought would be able and willing to try agile test practices, and wait for my opportunity.<br/>Things eventually slowed down enough for my team to have time to spike test automation solutions. <br/><em>Many of our applications are written in Tcl.</em> Tcl is a powerful scripting language which can lend itself to test automation, so we taught ourselves Tcl.</p>
<p><br/>We needed a way to load bulk data for volume testing, and one of the programmers helped out with a Tcl script; we took over maintenance for it and started expanding its functionality. <br/>Some of our test automation could be handled with fairly simple shell scripts.</p>
<p>Other automation requirements demanded more robust tools.</p>
<p>As we started down our own path of exploring test automation, the programmers started writing automated unit tests.<br/>I learned the hard way that being an advocate for XP or any kind of agile development doesn’t introduce change. <br/>Instead, you and your team need to identify what hurts and look at what XP or agile approaches could help heal that pain.</p>
<p><br/>By adopting XP-style planning and writing acceptance testing up front, my new development team has started exploring what agile practices might help them.</p>
<p><br/>Let’s look at what agile testing practices might help you, no matter what style of software development process you are using.<br/></p>The Nature of XP Testing, By André Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342062008-01-17T14:30:00.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">The biggest difference between XP projects and most ‘traditional’ software development projects is the concept of test-driven development. With XP, every chunk of code is covered by unit tests, which must all pass all the time. The absence of unit-level and regression bugs means that testers actually get to focus on their job: making sure the…</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">The biggest difference between XP projects and most ‘traditional’ software development projects is the concept of test-driven development. With XP, every chunk of code is covered by unit tests, which must all pass all the time. The absence of unit-level and regression bugs means that testers actually get to focus on their job: making sure the code does what the customer wanted. The acceptance tests define the level of quality the customer has specified (and paid for!)</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">Testers who are new to XP should keep in mind the XP values: communication, simplicity, feedback and courage. Courage may be the most important. As a tester, the idea of writing, automating and executing tests in speedy two or three week iterations, without the benefit of traditional requirements documents, can be daunting.</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">Testers need courage to let the customers make mistakes and learn from them. They need courage to determine the minimum testing that will prove the successful completion of a story. They need courage to ask their teammates to pair for test automation. They need courage to remind the team that we are all responsible for quality and testing. To bolster this courage, testers on XP teams should remind themselves that an XP tester is never alone – your team is always there to help you!</font></span></p>XP Tester Activities, By Andre Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342042008-01-17T14:29:06.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p><br></br>Here are some activities testers perform on XP teams.</p>
<p>•Negotiate quality with the customer (it’s not YOUR standard of quality, it’s what the customer desires and is willing to pay for!) <br></br>•Clarify stories, flush out hidden assumptions <br></br>•Enable accurate estimates for both programming and testing tasks <br></br>•Make sure the acceptance tests verify the quality specified by the customer <br></br>•Help the team automate tests <br></br>•Help the team produce testable code <br></br>•Form an…</p>
<p><br/>Here are some activities testers perform on XP teams.</p>
<p>•Negotiate quality with the customer (it’s not YOUR standard of quality, it’s what the customer desires and is willing to pay for!) <br/>•Clarify stories, flush out hidden assumptions <br/>•Enable accurate estimates for both programming and testing tasks <br/>•Make sure the acceptance tests verify the quality specified by the customer <br/>•Help the team automate tests <br/>•Help the team produce testable code <br/>•Form an integral part of the continuous feedback loop that keeps the team on track. <br/></p>How XP Testing is Different By Andre Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342022008-01-17T14:28:07.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">I found that XP testing was different in many ways from ‘traditional’ testing. The biggest difference is that on an XP project, the entire development team takes responsibility for quality. This means the whole team is responsible for all testing tasks, including acceptance test automation. When testers and programmers work together, the…</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">I found that XP testing was different in many ways from ‘traditional’ testing. The biggest difference is that on an XP project, the entire development team takes responsibility for quality. This means the whole team is responsible for all testing tasks, including acceptance test automation. When testers and programmers work together, the approaches to test automation can be pretty creative!</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><font style="BACKGROUND-COLOR: #d3dce3">As Ron Jeffries says, XP isn’t about ‘roles’, it’s about a tight integration of skills and behaviors. Testing is an integrated activity on an XP team. The development team needs continual feedback, with the customer expressing their needs in terms of tests, and programmers expressing design and code in terms of tests. On an XP team, the tester will play both the customer and programmer ‘roles’. She’ll focus on acceptance testing and work to transfer her testing and quality assurance skills to the rest of the team.</font></span></p>What is Agile Testing ? By Andre Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:342002008-01-17T14:27:18.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Historically, testing has always been about finding where the software breaks. In fact, when we talk about the qualities of a good tester we quote anecdotes about how "she was able to break my program in 3 minutes" or that "he found 27 bugs when running on Linux…</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Historically, testing has always been about finding where the software breaks. In fact, when we talk about the qualities of a good tester we quote anecdotes about how "she was able to break my program in 3 minutes" or that "he found 27 bugs when running on Linux with Apache". In other words, testing was about finding bugs. Testers would get rewarded for and take satisfaction from finding bugs.</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">It seems to me that agile testing is more about keeping the bugs out than finding the ones that are in there. The (automated) tests that we write are more about showing that the software works the way we expect than about finding where it breaks.</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">I am not claiming that agile testers should not attempt to find bugs nor that pre-agile testers never write tests to show what works. My claim is more modest. I am claiming that agile thinking has introduced a change in emphasis.</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Rather than make universal claims, I'll talk about my experience.</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">In my pre-agile days, developers would bang away on the code until they considered it done. Then they would throw the code over the wall to QA to find the bugs. QA's first task would be to find the places where it would break. They'd submit a whole bunch of bugs which developers would fix. This cycle would repeat until they couldn't find any more.</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US">Testing now happens closer to the developer in space and in time. Testers design tests to show that the software works as expected. The developers and testers work together to expand the set of cases that works. The bugs should never get introduced in the first place.</span></p>
<p class="MsoNormal" style="BACKGROUND: white; MARGIN: 0cm 0cm 4.5pt; LINE-HEIGHT: 140%"><span lang="EN-US" style="FONT-SIZE: 8.5pt; COLOR: #333333; LINE-HEIGHT: 140%; FONT-FAMILY: Verdana; mso-ansi-language: EN-US" xml:lang="EN-US"><strong>Agile thinking is *There Exists* thinking.</strong></span></p>XP Testing Without XP: Taking Advantage of Agile Testing Practices By Andre Vondrantag:knolstuff.com,2008-01-17:1781665:BlogPost:341982008-01-17T14:25:01.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p><span style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana"><font style="BACKGROUND-COLOR: #d3dce3">In the rocket-fast late 90s, I struggled with using traditional software process and testing practices on e-commerce applications. Then I read Kent Beck’s <i>Extreme Programming Explained</i> and had an amazing ‘aha’ moment. Ron Jeffries sums it up best:</font></span></p>
<p><font style="BACKGROUND-COLOR: #d3dce3"><i><span style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana">Extreme Programming is a…</span></i></font></p>
<p><span style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana"><font style="BACKGROUND-COLOR: #d3dce3">In the rocket-fast late 90s, I struggled with using traditional software process and testing practices on e-commerce applications. Then I read Kent Beck’s <i>Extreme Programming Explained</i> and had an amazing ‘aha’ moment. Ron Jeffries sums it up best:</font></span></p>
<p><font style="BACKGROUND-COLOR: #d3dce3"><i><span style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana">Extreme Programming is a discipline of software development based on values of simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation.</span></i> <span style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana">(<a href="http://www.xprogramming.com/"><font color="#0000FF">www.xprogramming.com</font></a>)</span></font></p>
<p><span style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana"><font style="BACKGROUND-COLOR: #d3dce3">Applying more discipline to traditional waterfall process had not helped my team. A new kind of discipline based on values such as communication, simplicity and feedback might be the answer! I immediately took an opportunity to join an XP team as a tester, and enjoyed working on XP teams for the next 18 months. Indeed, XP <i>did</i> allow us to produce high-quality e-commerce applications on time and on budget! Our customers were happy! We were happy! We struggled some with how I, as tester, could best contribute to the team, as the XP literature at the time didn’t say much on that subject. With the whole team focused on quality and testing, we came up with some great solutions. I was so excited about XP testing that I co-authored a book on it (<i>Testing Extreme Programming</i>, with Tip House).</font></span></p>Test Summary Report Identifier By André Vondran - QA Analysttag:knolstuff.com,2008-01-10:1781665:BlogPost:318382008-01-10T17:17:38.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Some type of unique company generated number to identify this summary report, its level</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">and the level of software that it is related to. Preferably the report level will be the same as…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Some type of unique company generated number to identify this summary report, its level</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">and the level of software that it is related to. Preferably the report level will be the same as the</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">related software level. The number may also identify whether the summary report is for the</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">entire project or a specific level of testing. This is to assist in coordinating software and testware</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">versions within configuration management.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Summary</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Identify all relevant support materials so that the reader of the report knows which version</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">and release of the project/software is being reported on. It may be particularly important to</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">identify the specific version of an external package used in the testing, especially if a new release</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">occurred during the test cycle and was not included. The version/release information should</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">match the information contained in the configuration management system and may include the</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">following elements.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><font face="Times New Roman"><b><span lang="EN-US" xml:lang="EN-US">Test Items</span></b> <span lang="EN-US" xml:lang="EN-US">– This should match the item definitions from the appropriate level test plan</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">that this report is covering. Any variance from the items specified in the test plan should</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">be identified. Elements from the features sections of the test plan (both included and</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">excluded) can also be included here or in a separate reference section.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><font face="Times New Roman"><b><span lang="EN-US" xml:lang="EN-US">Environment</span></b> <span lang="EN-US" xml:lang="EN-US">– The environment and any variances for that identified in the test plan</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">should be verified here to ensure that the correct test set-up was used. This will help</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">avoid confusion when the product is released to production and will ensure that the test</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">environment matches the destination platform.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><font face="Times New Roman"><b><span lang="EN-US" xml:lang="EN-US">References</span></b> <span lang="EN-US" xml:lang="EN-US">– Any documents that support this report and their location within the</span></font></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">configuration management system.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Variances</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Document any changes or deviations from those areas agreed on in the reference documents,</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">especially in areas that may cause concern to the group accepting the test results. Include</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">references to any supporting documentation that covers the reasons for the deviations.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">From Test Plans or Specifications</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Reasons for Deviations</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Support materials and documents</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Change requests</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Enhancement requests</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Incident reports (incident left in by intent)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Comprehensiveness Assessment</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Evaluation of the testing and test process in terms of the documented test objectives. This is</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">to assess the quality and effectiveness of testing so that assessment of the software can be viewed</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">correctly. Keep in mind that a coverage assessment only has meaning in relation to a known set</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">of initial test objectives.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Cover how effective testing was, and any weaknesses in the process, especially any</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">surprising trends and the causes of those deviations.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Evaluation of test coverage</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Total objectives (by Inventory category)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Objectives covered (depth and width)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Objectives omitted and reason for omission</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Identification of uncovered attributes</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Surprising trends and test process changes to cover them</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Summary of Results</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Report on the overall status of the incidents. Focus should be on trends and patterns in the</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">process and not on specific individuals or teams. Avoid pure numbers, as numbers due not really</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">provide insight as to the nature and cause of problems. The focus should be on costs, impacts,</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">and trends; including any <b>positive</b> trends. This is where you begin to set the stage for the</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">evaluation of the test process, the quality of the testing and the software quality and can include</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">areas such as:</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Total Incidents</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">By Severity and Priority</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">By cost/failure impact</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Defect Patterns</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Open or Unresolved incidents</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Evaluation</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Based on the evaluation of the testing as documented in sections three (3) through (6) assess</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">the quality of the software. This should be an objective assessment of the failure likelihood and</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">overall quality in terms of the criteria specified in the appropriate level test plan. Each item</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">identified in Section 2 under test items should be covered in the evaluation.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Include Good news as well. If the application tested out with high quality in some areas</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">(even though others were no so good) be sure to state that as well.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Limitations</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Incomplete or partial functions/features</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Dropped features (due to requirements change or defects)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Failure Likelihood</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">High or Medium risk areas</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Good quality areas or features</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Summary of Activities</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Cover the planned activities and the changes to those plans especially in areas where the</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">amount of actual effort greatly exceeded the planned effort. Include the reasons for the variances</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">and the possible impact on the testing staff. Major impacts to the testing staff will have possible</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">negative effects on follow-on projects or on the next project in line.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Staff time used</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Hours per day/week</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Elapsed time versus staff time</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Is staff working excess hours per week</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Costs – planned versus actual</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Variances and the reasons for the change</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Changes to the project scope and direction</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Requirements and design changes</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Surprising defect trends</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Loss of personnel (development, test, etc.)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span style="FONT-FAMILY: Symbol; mso-ansi-language: ES; mso-bidi-font-family: Symbol">· </span><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Test environment availability and accuracy</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Approvals</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Provide a list and signature block for each approving authority. This should match the list of</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; mso-layout-grid-align: none"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">names that approved the test plan in the first place. Those who agreed to the test plan need to</font></span></p>
<p><span lang="EN-US" style="FONT-SIZE: 12pt; FONT-FAMILY: 'Times New Roman'; mso-fareast-font-family: 'Times New Roman'; mso-ansi-language: EN-US; mso-fareast-language: ES; mso-bidi-language: AR-SA" xml:lang="EN-US">verify the results of the testing</span></p>
<p><strong>André Vondran</strong></p>
<p><strong>QA Analyst</strong></p>
<p><a href="http://www.aqasol.com.ar"><strong>www.aqasol.com.ar</strong></a></p>What Skills Performance Testers Need and How to Get Them? By André Vondran - QA Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316632008-01-09T20:57:03.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><font size="3">From time to time I see questions on different forums asking what skills are necessary for performance testers. There were pretty <b>interesting discussions</b>. Looks like most experts agree that performance testing requires more skills than just knowledge about how to create a script for a particular load testing tool. While it is still possible to imagine a…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><font size="3">From time to time I see questions on different forums asking what skills are necessary for performance testers. There were pretty <b>interesting discussions</b>. Looks like most experts agree that performance testing requires more skills than just knowledge about how to create a script for a particular load testing tool. While it is still possible to imagine a performance tester in a large corporation with deep specialization who only creates scripts and mechanically runs them while other performance experts monitor the system and analyze results, I don't see many perspectives neither for this person, nor for the approach. Systems become so complicated now that the sum of specialized expert views doesn't give the whole performance picture. <br/><br/>Thinking about skills necessary for performance testing I'd specify the following areas in addition to load testing per se:<br/><br/>• <i style="mso-bidi-font-style: normal">What is going on with the system?</i> Monitoring and Performance Analysis.<br/>• <i style="mso-bidi-font-style: normal">You got results. What if …?</i> Modeling and Capacity Planning.<br/>• <i style="mso-bidi-font-style: normal">We see the bottleneck. What to do?</i> Tuning and System Performance Engineering.<br/>• And write, present, communicate, organize all the time.<br/><br/>You probably need to know about tuning and have understanding of how applications should be designed to perform well (Software Performance Engineering). You don't need to be an expert, for example, in database tuning – most companies have DBAs for that – but you need to speak to a DBA in his language to coordinate efforts effectively. Or raise concerns about performance consequences of the current application design. Unfortunately it is not easy – you need to know enough to understand what is going on and communicate effectively.<br/><br/>The question is how to get such skills. Constant self-learning and gradually getting experience? Yes, of course, but that takes a lot of time. Moreover, many areas are pretty hard to jump into from a scratch. You need to get some basic understanding before you will be comfortable enough to learn further yourself. Go to a class? Definitely for a performance testing class and your main tool. But for many other different products you are working with? Usually there are several week-long performance-related classes for each product. They are developed for specialists making a living tuning this particular product. You don't have time to go to all classes and usually you don't need to dive so deep. Talk to an expert? Sure, if you find one around. Performance experts are sparse and busy, so you'd better have well-prepared question(s) – hard to do if you know little about the subject. <br/><br/>When you get well enough along the road you get into another trap. You already know enough that basic training won't be beneficial. Still there are almost no advanced classes at all for performance testers – when you go beyond basics, details of environments, tools, systems, application, etc. become so different that it makes no sense to make a class for specific combinations. You know areas where you need more information, you need to verify your approaches and practices against other experts, you need more advanced tips and tricks, and you need to find somebody you may discuss your problems with. <br/><br/>I believe that a good conference is a solution in both cases. Somebody digests information and presents it back to you. Not that it is an absolute ideal – quality of presentation and presenters vary – but probably the most effective way when you need to jump into many different topics. At the moment I am aware about one open and practical conference covering all performance-related aspects:</font> <a href="http://www.cmg.org/conference/cmg2007/"><b><span style="FONT-SIZE: 11pt"><font color="#FF8C00">Computer Measurement Group (CMG)</font></span></b></a><font size="3">. With over 150 sessions related to performance, it allows everybody to find something interesting. Let's look at</font> <a href="http://www.cmg.org/conference/cmg2007/"><b><span style="FONT-SIZE: 11pt"><font color="#FF8C00">the coming CMG conference</font></span></b></a> <font size="3">from a performance tester point of view. For Monitoring and Performance Analysis, you have five half-day classes under the CMG-T umbrella - real classes starting from the beginning and giving what you actually need to know. From leading experts in the field: one generic class, one class per major platform (Windows, Unix/Linux, mainframe), and one for network. Another class reviews what you need to know from statistics to be comfortable in the performance field:</font></span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/><br/>• Statistics for Performance Analysis & Capacity Planning by <i style="mso-bidi-font-style: normal">Ray Wicks, IBM<br/></i><br/></span><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><font size="3">Plus there are many presentation addressing specific environments or issues. A large variety of topics is covered from:</font></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/><br/>• Java Performance Analysis 401 by <i style="mso-bidi-font-style: normal">Peter Johnson, Unisys <br/></i><br/>• Windows Server 2008 Performance, Scalability and Tools by <i style="mso-bidi-font-style: normal">Bradley M. Waters, Microsoft</i> <br/><br/></span><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><font size="3">to very specific topics like Capacity Planning is a core area of CMG so there are many presentations investigating the subject in detail. Starting with the first part of the keynote presentation:<br/><br/>• Is Capacity Planning Still Relevant? by Jerred Ruble, TeamQuest Corporation <br/><br/>A caution here is that many modeling and capacity planning presentations are pretty advanced for an unprepared listener. While I strongly believe that a performance tester must understand the basic concepts of modeling and capacity planning (to be able to do "back of the envelope" calculations), serious mathematical modeling may be pretty complicated and is rather optional for performance testers.<br/><br/>Another CMG focus track is Software Performance Engineering (SPE). There is a great tutorial to start from:</font></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/><br/>• Software Performance Engineering: A Tutorial Introduction by <i style="mso-bidi-font-style: normal">Dr. Lloyd G. Williams, PerfX</i> and <i style="mso-bidi-font-style: normal">Connie U. Smith, Performance Engineering Services</i> <br/><br/></span><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><font size="3">The following presentations are directly related to performance testing:<br/></font></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/>• How to Select Significant Workloads in Performance Models by <i style="mso-bidi-font-style: normal">Paolo Cremonesi, Politecnico di Milano<br/></i><br/>• Multiple Dimensions of Performance Requirements by <i style="mso-bidi-font-style: normal">Alexander Podelko, Oracle<br/></i><br/></span><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><font size="3">One more great type of presentation involves overviews of different technologies for "performance" professionals: some insights that you need to know and probably won't find in marketing materials. There are so many new buzzwords that you can not go to training for each of them (and shouldn't - usually at least 95% of what is included in training is irrelevant to performance testing). One hour presentation discussing what this new thing may mean for you can give you a pretty good overview, good enough for what you are doing. </font></span></p>
<p/><p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar/">www.aqasol.com.ar</a></p>What is Software Testing? By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316612008-01-09T20:55:37.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Defect can be caused by a flaw in the application software or by a flaw in the application specification. For example, unexpected (incorrect) results can be from errors made during the construction phase, or from an algorithm incorrectly defined in the specification. Testing is commonly assumed to mean executing software and…</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Defect can be caused by a flaw in the application software or by a flaw in the application specification. For example, unexpected (incorrect) results can be from errors made during the construction phase, or from an algorithm incorrectly defined in the specification. Testing is commonly assumed to mean executing software and finding errors. This type of testing is known as dynamic testing, and while valid, it is not the most effective way of testing. Static testing, the review, inspection and validation of development requirements, is the most effective and cost efficient way of testing. A structured approach to testing should use both dynamic and static testing techniques.</span><span><br/><br/><font size="3"><strong><u><em>Testing and Quality Assurance</em></u></strong></font></span><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></span><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">What is the relationship between testing and Software Quality Assurance (SQA)? An application that meets its requirements totally can be said to exhibit quality. Quality is not based on a subjective assessment but rather on a clearly demonstrable, and measurable, basis. Quality Assurance and Quality Control are not the same. Quality Control is a process directed at validating that a specific deliverable meets standards, is error free, and is the best deliverable that can be produced. It is a responsibility internal to the team. QA, on the other hand, is a review with a goal of improving the process as well as the deliverable. QA is often an external process. QA is an effective approach to producing a high quality product. One aspect is the process of objectively reviewing project deliverables and the processes that produce them (including testing), to identify defects, and then making recommendations for improvement based on the reviews. The end result is the assurance that the system and application is of high quality, and that the process is working. The achievement of quality goals is well within reach when organizational strategies are used in the testing process. From the client's perspective, an application's quality is high if it meets their expectations.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3"><strong><u>What is the difference between a bug, a defect, and an error?</u></strong></font></span></i></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">According to the British norm BS 7925-1: bug--generic term for fault, failure, error, human action that produces an incorrect result.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Error:</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">programmatically mistake leads to error.<br/><b style="mso-bidi-font-weight: normal">Bug:</b> Deviation from the expected result. <br/><b style="mso-bidi-font-weight: normal">Defect:</b> Problem in algorithm leads to failure.<br/><b style="mso-bidi-font-weight: normal">Failure:</b> Result of any of the above.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Compare those to these arbitrary definitions:</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/><b style="mso-bidi-font-weight: normal">Error:</b> When we get the wrong output i.e. syntax error, logical error<br/><b style="mso-bidi-font-weight: normal">Fault:</b> When everything is correct but we are not able to get a result<br/><b style="mso-bidi-font-weight: normal">Failure:</b> We are not able to insert any input</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3"><strong><u>How to Write a Fully Effective Bug Report</u></strong></font></span></i></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">To write a fully effective report you must: <br/>- Explain how to reproduce the problem - Analyze the error so you can describe it in a minimum number of steps. <br/>- Write a report that is complete and easy to understand.<br/><br/>Write bug reports immediately; the longer you wait between finding the problem and reporting it, the more likely it is the description will be incomplete, the problem not reproducible, or simply forgotten.<br/><br/>Writing a one-line report summary (Bug's report title) is an art. You must master it. Summaries help everyone quickly review outstanding problems and find individual reports. The summary line is the most frequently and carefully read part of the report. When a summary makes a problem sound less severe than it is, managers are more likely to defer it. Alternatively, if your summaries make problems sound more severe than they are, you will gain a reputation for alarmism. Don't use the same summary for two different reports, even if they are similar. The summary line should describe only the problem, not the replication steps. Don't run the summary into the description (Steps to reproduce) as they will usually be printed independently of each other in reports.<br/><br/>Ideally you should be able to write this clearly enough for a developer to reproduce and fix the problem, and another QA engineer to verify the fix without them having to go back to you, the author, for more information. It is much better to over communicate in this field than say too little. Of course it is ideal if the problem is reproducible and you can write down those steps. But if you can't reproduce a bug, and try and try and still can't reproduce it, admit it and write the report anyway. A good programmer can often track down an irreproducible problem from a careful description. For a good discussion on analyzing problems and making them reproducible, see Chapter 5 of <a href="http://www.amazon.com/exec/obidos/ASIN/0471358460/sqatestercom-20" target="_blank"><font color="#330066">Testing Computer Software</font></a> by Cem Kaner.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">The most controversial thing in a bug report is often the bug Impacts: <i style="mso-bidi-font-style: normal">Low</i>, <i style="mso-bidi-font-style: normal">Medium</i>, High, and <i style="mso-bidi-font-style: normal">Urgent</i>. Report should show the priority which you, the bug submitter, believes to be appropriate and does not get changed.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/></span></b><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3">Software Testing 10 Rules</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-fareast-font-family: 'Book Antiqua'; mso-bidi-font-family: 'Book Antiqua'" xml:lang="EN-US"><span style="mso-list: Ignore">1. </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Test early and test often.<br/><br/>2. Integrate the application development and testing life cycles. You'll get better results and you won't have to mediate between two armed camps in your IT shop.<br/><br/>3. Formalize a testing methodology; you'll test everything the same way and you'll get uniform results.<br/><br/>4. Develop a comprehensive test plan; it forms the basis for the testing methodology.<br/><br/>5. Use both static and dynamic testing.<br/><br/>6. Define your expected results.<br/><br/>7. Understand the business reason behind the application. You'll write a better application and better testing scripts.<br/><br/>8. Use multiple levels and types of testing (regression, systems, integration, stress and load).<br/><br/>9. Review and inspect the work, it will lower costs.<br/><br/>10. Don't let your programmers check their own work; they'll miss their own errors.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3">Bug Impacts</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Low impact</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/>This is for Minor problems, such as failures at extreme boundary conditions that are unlikely to occur in normal use, or minor errors in layout/formatting. These problems do not impact use of the product in any substantive way.<br/></span><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/></span><a name="Medium" id="Medium"></a><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Medium impact</span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/>This is a problem that a) Effects a more isolated piece of functionality. b) Occurs only at certain boundary conditions. c) Has a workaround (where "don't do that" might be an acceptable answer to the user). d) Occurs only at one or two customers. or e) Is very intermittent<br/></span><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/></span></b><a name="High" id="High"></a><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">High impact</span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/>This should be used for only serious problems, effecting many sites, with no workaround. Frequent or reproducible crashes/core dumps/GPFs would fall in this category, as would major functionality not working.<br/></span><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/></span><a name="Urgent" id="Urgent"></a><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Urgent impact</span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/>This should be reserved for only the most catastrophic of problems. Data corruption, complete inability to use the product at almost any site, etc. For released products, an urgent bug would imply that shipping of the product should stop immediately, until the problem is resolved.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3">Bug Report Components</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Report number:</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Unique number given to a bug.<br/><b style="mso-bidi-font-weight: normal">Program / module being tested:</b> The name of a program or module that being tested<br/><b style="mso-bidi-font-weight: normal">Version & release number:</b> The version of the product that you are testing.<br/><b style="mso-bidi-font-weight: normal">Problem Summary:</b> (data entry field that's one line) precise to what the problem is. <br/><b style="mso-bidi-font-weight: normal">Report Type:</b> Describes the type of problem found, for example it could be software or hardware bug. <br/><b style="mso-bidi-font-weight: normal">Severity:</b> Normally, how you view the bug. <b style="mso-bidi-font-weight: normal">Various levels of severity:</b> Low - Medium - High - Urgent<br/><b style="mso-bidi-font-weight: normal">Environment:</b> Environment in which the bug is found.<br/><b style="mso-bidi-font-weight: normal">Detailed Description:</b> Detailed description of the bug that is found<br/><b style="mso-bidi-font-weight: normal">How to reproduce:</b> Detailed description of how to reproduce the bug.<br/><b style="mso-bidi-font-weight: normal">Reported by:</b> The name of person who writes the report.<br/><b style="mso-bidi-font-weight: normal">Assigned to developer:</b> The name of developer who assigned to fixed the bug.<br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Status:</span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><br/><u>Open:</u> The status of bug when it entered.<br/><u>Fixed / feedback:</u> The status of the bug when it fixed.<br/><u>Closed:</u> The status of the bug when verified.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">(Bug can be only closed by QA person. Usually, the problem is closed by QA manager.)<br/><u>Deferred:</u> The status of the bug when it postponed.<br/><u>User error:</u> The status of the bug when user made an error.<br/><u>Not a bug:</u> The status of the bug when it is not a bug.<br/><br/><b style="mso-bidi-font-weight: normal">Priority:</b> Assigned by the project manager who asks the programmers to fix bugs in priority order.<br/><b style="mso-bidi-font-weight: normal">Resolution:</b> Defines the current status of the problem. There are four types of resolution such as deferred, not a problem, will not fix, and as designed</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3">Defects Severity and Priority</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Question:</span></i></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">One question on the defects that we raise. We are supposed to give a<br/>severity and a priority to it. Now, the severity can be Major, Minor or<br/>Trivial and the Priority can be 1, 2 or 3 (with 1 being a high priority<br/>defect).<br/>My question is - why do we need two parameters, severity and priority, for a defect Can't we do only with one?</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Answer (1):</span></i></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">It depends entirely on the size of the company. Severity tells us how bad the defect is. Priority tells us how soon it is desired to fix the problem.<br/>In some companies, the defect reporter sets the severity and the triage team or product management sets the priority. In a small company, or project (or product), particularly where there aren't many defects to track, you can expect you don't really need both since a high severity defect is also a high priority defect. But in a large company, and particularly where there are many defects, using both is a form of risk management.<br/><br/>Major would be 1 and Trivial would be 3. You can add or multiply the two values together (there is only a small difference in the outcome) and then use the event's risk value to determine how you should address the problem. The lower values must be addressed and the higher values can wait.<br/><br/>I discovered a new method for Risk Assessment. It is based on a military standard, MIL-STD-882. If you want a copy of the current version, search for MIL-STD-882D using Google or Yahoo! The main area of interest is section A.4.4.3 and its children where they indicate the Assessment of mishap risk.<br/>They use a four-point severity rating (rather than three): Catastrophic; Critical; Marginal; Negligible. They then use a five-point (rather than three) probability rating: Frequent; Probable; Occasional; Remote; Improbable. Then rather than using a mathematical calculation to determine a risk level, they use a predefined chart.<br/><br/><i>Blocker</i>: This bug prevents developers from testing or developing the software.<br/><i>Critical</i>: The software crashes, hangs, or causes you to lose data.<br/><i>Major</i>: A major feature is broken.<br/><i>Normal</i>: It's a bug that should be fixed.<br/><i>Minor</i>: Minor loss of function, and there's an easy work around.0.<br/><i>Trivial</i>: A cosmetic problem, such as a misspelled word or misaligned text.<br/>Enhancement: Request for new feature or enhancement.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Answer (2):</span></i></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Severity Levels can be defined as follow: <i>S1</i> - Urgent/Showstopper. Like system crash or error message forcing to close the window. Tester's ability to operate the system either totally (System Down), or<br/>almost totally, affected. A major area of the users system is affected by the incident and it is significant to business processes. <i>S2</i> - Medium/Workaround. Exist like when a problem is required in the specs but tester can go on with testing. Incident affects an area of functionality but there is a work-around which negates impact to business process. This is a problem that:<br/>a) Affects a more isolated piece of functionality.<br/>b) Occurs only at certain boundary conditions.<br/>c) Has a workaround (where "don't do that" might be an acceptable answer to the user).<br/>d) Occurs only at one or two customers. or is intermittent<br/><i>S3</i> - Low. This is for minor problems, such as failures at extreme boundary conditions that are unlikely to occur in normal use, or minor errors in layout/formatting. Problems do not impact use of the product in any substantive way. These are incidents that are cosmetic in nature and of no or very low impact to business processes.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="COLOR: black; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3">Documentation Tips</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Important areas of "black box" when QA engineer write test plans:<br/>* Acceptance test (into testing)<br/>* Data flow and integrity<br/>* Configuration and compatibility<br/>* Stress test<br/>* Regressions<br/>* Performance<br/>* Potential bugs<br/>* Beta tests<br/>* Release tests<br/>* Utility<br/>* User interfaces</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">It is a good practice to have developers review test cases after they have been written. Knowing design of the specific feature or product in whole, developers can give you a valuable feedback on what is missing from test cases and should be added, what areas to pay more attention while testing and even how to apply it Test cases should be updated based on gotten feedback.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US"><font size="3">How to create a Test Plan without docs?</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">1. Try to break up your huge application into modules that are functionally independent.<br/>2. Within each module you start with the functions one by one.<br/>3. For a simple function write all possible test cases that arise to be tested, while using the application as there are no specs.<br/>4. In this way you could complete one function and in turn whole application.<br/>5. To prepare test cases or plan make use of Excel sheet. Each sheet will define each function within the module. This is best way to organize the test cases.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: Arial" xml:lang="EN-US">Test Plan Sample</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman" size="3">1. Introduction</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="1_doc_descr"></a><b><i style="mso-bidi-font-style: normal"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman" size="3">Description of this Document</font></span></i><span lang="EN-US" xml:lang="EN-US"><br/></span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">This document is a Test Plan for the -Project name-, produced by Quality Assurance. It describes the testing strategy and approach to testing QA will use to validate the quality of this product prior to release. It also contains various resources required for the successful completion of this project. <br/><br/>The focus of the -Project name- is to support those new features that will allow easier development, deployment and maintenance of solutions built upon the -Project name-. Those features include:</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of the features]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">This release of the -Project name- will also include legacy bug fixing, and redesigning or including missing functionality from previous release</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of the features]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The following implementations were made:</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List and description of implementations made]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="1_rel_doc"></a><b><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Related Documents</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of related documents such as: Functional Specifications, Design Specifications]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="1_schedule"></a><b><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Schedule and Milestones</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[Schedule information QA testing estimates]<br/><br/><a name="2_req"></a><b><i style="mso-bidi-font-style: normal">2. Resource Requirements</i></b></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="2_hardware"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Hardware</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of hardware requirements]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="2_software"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Software</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of software requirements: primary and secondary OS]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="2_tools"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Test Tools</span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/>Apart from manual tests, the following tools will be used:</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">-<br/><br/><a name="2_staffing"></a><b>Staffing</b></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="2_responsibilities"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Responsibilities<br/></span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of QA team members and there responsibilities]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="2_training"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Training<br/></span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of training's required]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="3_to_test"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">3. Features To Be Tested / Test Approach</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of the features to be tested]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="3_test_media"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Media Verification<br/></span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[The process will include installing all possible products from the media and subjecting them to basic sanity testing.]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="4_not_test"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">4. Features Not To Be Tested</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of the features not to be tested]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="5_deliverables"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">5. Test Deliverables</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of the test cases/matrices or there location]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">[List of the features to be automated]</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="6_risks"></a><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">6. Dependencies/Risks</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Dependencies</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Risks<br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/><a name="7_mec"></a></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">7. Milestone Criteria</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US"><font size="3">Key QA Documents</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 54.0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial; mso-fareast-font-family: Arial" xml:lang="EN-US"><span style="mso-list: Ignore">I. </span></span><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">PRAD</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><br/>Product Requirement Analysis Document is the document prepared/reviewed by marketing, sales, and technical product managers. This document defines the requirements for the product, the "What". It is used by the developer to build his/her functional specification and used by QA as a reference for the first draft of the Test Strategy.<br/><br/><b style="mso-bidi-font-weight: normal">II. Functional Specification</b><br/><br/>The functional specification is the "How" of the product. The functional specification identifies how new features will be implemented. This document includes items such as what database tables a particular search will query. This document is critical to QA because it is used to build the Test Plan.<br/><br/>QA is often involved in reviewing the functional specification for clarity and helping to define the business rules.<br/><br/><b style="mso-bidi-font-weight: normal">III. Test Strategy</b><br/><br/>The Test Strategy is the first document QA should prepare for any project. This is a living document that should be maintained/updated throughout the project. The first draft should be completed upon approval of the PRAD and sent to the developer and technical product manager for review. <br/><br/>The Test Strategy is a high-level document that details the approach QA will follow in testing the given product. This document can vary based on the project, but all strategies should include the following criteria:<br/>· Project Overview - What is the project. <br/>· Project Scope - What are the core components of the product to be tested<br/>· Testing - This section defines the test methodology to be used, the types of testing to be executed (GUI, Functional, etc.), how testing will be prioritized, testing that will and will not be done and the associated risks. This section should also outline the system configurations that will be tested and the tester assignments for the project.<br/>· Completion Criteria - These are the objective criteria upon which the team will decide the product is ready for release<br/>· Schedule - This should define the schedule for the project and include completion dates for the PRAD, Functional Spec, and Test Strategy etc. The schedule section should include build delivery dates, release dates and the dates for the Readiness Review, QA Process Review, and Release Board Meetings.<br/>· Materials Consulted - Identify the documents used to prepare the test strategy<br/>· Test Setup - This section should identify all hardware/software, personnel pre-requisites for testing. This section should also identify any areas that will not be tested (such as 3rd party application compatibility.)<br/><br/><b style="mso-bidi-font-weight: normal">IV. Test Matrix (Test Plan)</b><br/><br/>The Test Matrix is the Excel template that identifies the test types (GUI, Functional etc.), the test suites within each type, and the test categories to be tested. This matrix also prioritizes test categories and provides reporting on test coverage.<br/>· Test Summary report<br/>· Test Suite Risk Coverage report<br/><br/>Upon completion of the functional specification and test strategy, QA begins building the master test matrix. This is a living document and can change over the course of the project as testers create new test categories or remove non-relevant areas. Ideally, a master matrix need only be adjusted to include near feature areas or enhancements from release to release on a given product line.<br/><br/><b style="mso-bidi-font-weight: normal">V. Test Cases</b><br/><br/>As testers build the Master Matrix, they also build their individual test cases. These are the specific functions testers must verify within each test category to qualify the feature. A test case is identified by ID number and prioritized. Each test case has the following criteria:<br/>· Purpose - Reason for the test case<br/>· Steps - A logical sequence of steps the tester must follow to execute the test case<br/>· Expected Results - The expected result of the test case <br/>· Actual Result - What actually happened when the test case was executed<br/>· Status - Identifies whether the test case was passed, failed, blocked or skipped. <br/>· Pass - Actual result matched expected result<br/>· Failed - Bug discovered that represents a failure of the feature<br/>· Blocked - Tester could not execute the test case because of bug<br/>· Skipped - Test case was not executed this round<br/>· Bug ID - If the test case was failed, identify the bug number of the resulting bug.<br/><br/><b style="mso-bidi-font-weight: normal">VI. Test Results by Build</b><br/><br/>Once QA begins testing, it is incumbent upon them to provide results on a consistent basis to developers and the technical product manager. This is done in two ways: A completed Test Matrix for each build and a Results Summary document. <br/><br/>For each test cycle, testers should fill in a copy of the project's Master Matrix. This will create the associated Test Coverage reports automatically (Test Coverage by Type and Test Coverage by Risk/Priority). This should be posted in a place that necessary individuals can access the information.<br/>Since the full Matrix is large and not easily read, it is also recommended that you create a short Results Summary that highlights key information. A Results Summary should include the following:<br/>· Build Number<br/>· Database Version Number<br/>· Install Paths (If applicable)<br/>· Testers<br/>· Scheduled Build Delivery Date<br/>· Actual Build Delivery Date<br/>· Test Start Date<br/>· Scope - What type of testing was planned for this build? For example, was it a partial build? A full-regression build? Scope should identify areas tested and areas not tested.<br/>· Issues - This section should identify any problems that hampered testing, represent a trend toward a specific problem area, or are causing the project to slip. For example, in this section you would note if the build was delivered late and why and what its impact was on testing.<br/>· Statistics - In this section, you can note things such as number of bugs found during the cycle, number of bugs closed during the cycle etc.<br/><br/><b style="mso-bidi-font-weight: normal">VII. Release Package</b><br/><br/>The Release Package is the final document QA prepares. This is the compilation of all previous documents and a release recommendation. Each release package will vary by team and project, but they should all include the following information:<br/>· Project Overview - This is a synopsis of the project, its scope, any problems encountered during the testing cycle and QA's recommendation to release or not release. The overview should be a "response" to the test strategy and note areas where the strategy was successful, areas where the strategy had to be revised etc.<br/>The project overview is also the place for QA to call out any suggestions for process improvements in the next project cycle.<br/>Think of the Test Strategy and the Project Overview as "Project bookends".<br/>· Project PRAD - This is the Product Requirements Analysis Document, which defines what functionality was approved for inclusion in the project. If there was no PRAD for the project, it should be clearly noted in the Project Overview. The consequences of an absent PRAD should also be noted.<br/>· Functional Specification - The document that defines how functionality will be implemented. If there was no functional specification, it should be clearly noted in the Project Overview. The consequences of an absent Functional Specification should also be noted.<br/>· Test Strategy - The document outlining QA's process for testing the application.<br/>· Results Summaries - The results summaries identify the results of each round of testing. These should be accompanied in the Release Package by the corresponding reports for Test Coverage by Test Type and Test Coverage by Risk Type/Priority from the corresponding completed Test Matrix for each build. In addition, it is recommended that you include the full Test Matrix results from the test cycle designated as Full Regression.<br/>· Known Issues Document - This document is primarily for Technical Support. This document identifies workarounds, issues development is aware of but has chosen not to correct, and potential problem areas for clients.<br/>· Installation Instruction - If your product must be installed as the client site, it is recommended to include the Installation Guide and any related documentation as part of the release package.<br/>· <i style="mso-bidi-font-style: normal">Open Defects</i> - The list of defects remaining in the defect tracking system with a status of Open. Technical Support has access to the system, so a report noting the defect ID, the problem area, and title should be sufficient.<br/>· <i style="mso-bidi-font-style: normal">Deferred Defects</i> - The list of defects remaining in the defect tracking system with a status of deferred. Deferred means the technical product manager has decided not to address the issue with the current release.<br/>· <i style="mso-bidi-font-style: normal">Pending Defects</i> - The list of defects remaining in the defect tracking system with a status of pending. Pending refers to any defect waiting on a decision from a technical product manager before a developer addresses the problem.<br/>· <i style="mso-bidi-font-style: normal">Fixed Defects</i> - The list of defects waiting for verification by QA.<br/>· <i style="mso-bidi-font-style: normal">Closed Defects</i> - The list of defects verified as fixed by QA during the project cycle.<br/>The Release Package is compiled in anticipation of the Readiness Review meeting. It is reviewed by the QA Process Manager during the QA Process Review Meeting and is provided to the Release Board and Technical Support.<br/>· <b style="mso-bidi-font-weight: normal">Readiness Review Meeting:</b> The Readiness Review meeting is a team meeting between the technical product manager, project developers and QA. This is the meeting in which the team assesses the readiness of the product for release.<br/>This meeting should occur prior to the delivery of the Gold Candidate build. The exact timing will vary by team and project, but the discussion must be held far enough in advance of the scheduled release date so that there is sufficient time to warn executive management of a potential delay in the release.<br/>The technical product manager or lead QA may schedule this meeting. <br/><br/><b style="mso-bidi-font-weight: normal">· QA Process Review Meeting:</b> The QA Process Review Meeting is meeting between the QA Process Manager and the QA staff on the given project. The intent of this meeting is to review how well or not well process was followed during the project cycle. <br/>This is the opportunity for QA to discuss any problems encountered during the cycle that impacted their ability to test effectively. This is also the opportunity to review the process as whole and discuss areas for improvement.<br/>After this meeting, the QA Process Manager will give a recommendation as to whether enough of the process was followed to ensure a quality product and thus allow a release.<br/>This meeting should take place after the Readiness Review meeting. It should be scheduled by the lead QA on the project.<br/><br/><b style="mso-bidi-font-weight: normal">· Release Board Meeting:</b> This meeting is for the technical product manager and senior executives to discuss the status of the product and the teams release recommendations. If the results of the Readiness meeting and QA Process Review meeting are positive, this meeting may be waived.<br/>The technical product manager is responsible for scheduling this meeting. <br/>This meeting is the final check before a product is released. <br/>Due to rapid product development cycles, it is rare that QA receives completed PRADs and Functional Specifications before they begin working on the Test Strategy, Test Matrix, and Test Cases. This work is usually done in parallel. <br/>Testers may begin working on the Test Strategy based on partial PRADs or confirmation from the technical product manager as to what is expected to be in the next release. This is usually enough to draft out a high -level strategy outlining immediate resource needs, potential problem areas, and a tentative schedule. <br/>The Test Strategy is then updated once the PRAD is approved, and again when the functional specifications are complete enough to provide management with a committed schedule. All drafts of the test strategy should be provided to the technical product manager and it is QA's responsibility to ensure that information provided in the document (such as potential resource problems) is clearly understood.<br/>If the anticipated release does not represent a new product line, testers can begin the Master Test Matrix and test cases at the same time the project's PRAD is being finalized. Testers can build and/or refine test cases for the new functionality as the functional specification is defined. Testers often contribute to and are expected to be involved in reviewing the functional specification.<br/>The results summary document should be prepared at the end of each test cycle and distributed to developers and the technical product manager. It is designed more to inform interested parties on the status of testing and possible impact to the overall project cycle. <br/><br/>The release package is prepared during the last test cycle for the readiness review meeting. <br/>Test Strategy Template<br/><br/>QA Test Strategy: [Product and Version] <br/><br/>[Document Version history in format MM-DD-YYYY]<br/><br/><b style="mso-bidi-font-weight: normal">1.0 PROJECT OVERVIEW</b><br/><br/>[Brief description of project] <br/><br/><b style="mso-bidi-font-weight: normal">1.2 PROJECT SCOPE</b><br/><br/>[More detailed description of project detailing functionality to be included] <br/><br/><b style="mso-bidi-font-weight: normal">2.0 MATERIALS CONSULTED</b><br/><br/>[Identify all documentation used to build the test strategy]<br/><br/><b style="mso-bidi-font-weight: normal">3.0 TESTING</b><br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· CRITICAL FOCUS AREAS</u></i></b><br/><br/>[Areas identified by developers as potential problems above and beyond specific feature enhancements or new functionality already given priority 1 status by QA]<br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· INSTALLATION:</u></i></b><br/><br/>[Installation paths to be qualified by QA. Not all products require installation testing. However, those that do often have myriad installation paths. Due to time and resource constraints, QA must prioritize. Decisions on which installation paths to test should be made in cooperation with the technical product manager. Paths not slated for testing should also be identified here.]<br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· GUI</u></i></b><br/><br/>[Define what if any specific GUI testing will be done]<br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· FUNCTIONAL</u></i></b><br/><br/>[Define the functionality to be tested and how it will be prioritized]<br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u><br/>· INTEGRATION</u></i></b><br/><br/>[Define the potential points of integration with other MediaMap products and how they will be prioritized and tested]<br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· SECURITY</u></i></b><br/><br/>[Define how security issues will be tested and prioritized]<br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· PERFORMANCE</u></i></b><br/><br/>[Define what if any performance testing will be done and its priority]<br/><br/><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u>· FAILURE RECOVERY</u></i></b><br/><br/>[Define what if any failure recovery testing will be done and its priority]<br/><br/><b style="mso-bidi-font-weight: normal">3.1 TECHNIQUE</b><br/><br/>· [Technique used for testing. Automation vs. Manual]<br/><br/><b style="mso-bidi-font-weight: normal">3.2 METHODOLOGY</b><br/><br/>[Define how testers will go about testing the product. This is where you outline your core strategy. Include in this section anything from tester assignments to tables showing the operating systems and browsers the team will qualify. It is also important to identify any testing limitations and risks] <br/><br/><b style="mso-bidi-font-weight: normal">4.0 TEST SET-UP<br/><br/>4.1 TEST PRE-REQUISITES</b><br/><br/>[Any non-software or hardware related item QA needs to test the product. For example, this section should identify contact and test account information for 3rd party vendors]<br/><br/><b style="mso-bidi-font-weight: normal">4.2 HARDWARE</b><br/><br/>QA has the following machines available for testing:<br/><br/>Workstations: Servers:<br/>[Include processor, chip, and memory and disk space]<br/><br/>Other:<br/>[Identify any other hardware needed such as modems etc.]<br/><br/><b style="mso-bidi-font-weight: normal">4.3 SOFTWARE</b><br/><br/>[Identify all those software applications QA will qualify with the product and those QA will not qualify. For example, this is where you would list the browsers to be qualified. It is also important to identify what will not be qualified (for example, not testing with Windows 2000)]<br/><b style="mso-bidi-font-weight: normal"><br/>4.4 PERSONNEL</b><br/><br/>[Identify which testers are assigned to the project and who will test what. It is also important to identify who is responsible for the creation of the test strategy, test plan, test cases, release package, documentation review etc.]<br/><br/><b style="mso-bidi-font-weight: normal">5.0 COMPLETION CRITERIA</b> <br/><br/>[Identify how you will measure whether the product is ready for release. For example, what is the acceptable level of defects in terms of severity, priority, and volume?]<br/><br/><b style="mso-bidi-font-weight: normal">6.0 SCHEDULE</b><br/><br/>6.1 Project Schedule <br/>· PRD Review completed by [MM-DD-YYYY] - [STATUS] <br/>· Functional Specification completed [MM-DD-YYYY] - [STATUS]<br/>· Release Date approved by [MM-DD-YYYY] - [STATUS]<br/>· Test Strategy completed by [MM-DD-YYYY] - [STATUS]<br/>· Core Test Plan (functional) completed by [MM-DD-YYYY] - [STATUS]<br/>· Readiness Meeting - [STATUS]<br/>· QA Process Review Meeting - [STATUS] <br/>· Release Board Meeting - [STATUS]<br/>· Release on [MM-DD-YYYY] - [STATUS]<br/><br/>6.2 Build Schedule<br/>· Receive first build on [MM-DD-YYYY] - [STATUS] <br/>· Receive second build on [MM-DD-YYYY] - [STATUS] <br/>· Receive third build on [MM-DD-YYYY] - [STATUS] <br/>· Receive fourth build on [MM-DD-YYYY] - [STATUS]<br/>· Receive Code Freeze Build on [MM-DD-YYYY] - [STATUS]<br/>· Receive Full Regression Build on [MM-DD-YYYY] - [STATUS]<br/>· Receive Gold Candidate Build on [MM-DD-YYYY] - [STATUS]<br/>· Final Release on [MM-DD-YYYY] - [STATUS]<br/><br/><b style="mso-bidi-font-weight: normal">7.0 QA Test Matrix and Test Cases:</b></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US"><font size="3">What are Test Cases?</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Question:</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">What are Test Cases, Test Suites, Test Scripts, and Test Scenarios?<br/><br/><b>Answer:</b> A test case is a specific set of steps, it has expected result, along with various additional pieces of information. These optional fields are a test case ID, test step or order of execution number, related requirement(s), depth, test category, author, and check boxes for whether the test is automatable and has been automated. Larger test cases may also contain prerequisite states or steps, and descriptions. A test case should also contain a place for the actual result. These steps can be stored in a word processor document, spreadsheet, database or other common repository. In a database system, you may also be able to see past test results and who generated the results and the system configuration used to generate those results.<br/><br/>The most common term for a collection of test cases is a test suite. The test suite often also contains more detailed instructions or goals for each collection of test cases. It definitely contains a section where the tester identifies the system configuration used during testing. A group of test cases may also contain prerequisite states or steps, and descriptions of the following tests.<br/><br/>Collections of test cases are sometimes incorrectly termed a test plan. They may also be called a test script, or even a test scenario. A test plan is the approach that will be used to test the system, not the individual tests.<br/>Most companies that use automated testing will call the code that is used their test scripts.<br/><br/>A scenario test is a test based on a hypothetical story used to help a<br/>person think through a complex problem or system. They can be as simple as a diagram for a testing environment or they could be a description written in prose. The ideal scenario test has five key characteristics. It is (a) a story that is (b) motivating, (c) credible, (d) complex, and (e) easy to evaluate. They are usually different from test cases in that test cases are single steps and scenarios cover a number of steps. Test suites and scenarios can be used in concert for complete system tests. See<br/></span><span lang="EN-US" style="FONT-SIZE: 10pt" xml:lang="EN-US"><a href="http://www.kaner.com/pdfs/ScenarioIntroVer4.pdf" target="_blank"><span style="FONT-FAMILY: Arial"><font color="#330066">http://www.kaner.com/pdfs/ScenarioIntroVer4.pdf</font></span></a></span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">and<br/></span><span lang="EN-US" style="FONT-SIZE: 10pt" xml:lang="EN-US"><a href="http://www.kaner.com/pdfs/ScenarioSTQE.pdf" target="_blank"><span style="FONT-FAMILY: Arial"><font color="#330066">http://www.kaner.com/pdfs/ScenarioSTQE.pdf</font></span></a></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p/><p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></p>
<p/>Testing Risk Assessment By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316592008-01-09T20:53:03.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<span lang="EN-US" style="FONT-SIZE: 10pt; COLOR: black; FONT-FAMILY: Arial" xml:lang="EN-US"><br />
</span><p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; COLOR: black; FONT-FAMILY: Arial" xml:lang="EN-US"><tt><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial; mso-ansi-font-size: 12.0pt; mso-bidi-font-size: 12.0pt" xml:lang="EN-US"><em>The purpose of business risk analysis during software testing is identifying high-risk…</em></span></tt></span></p>
<span lang="EN-US" style="FONT-SIZE: 10pt; COLOR: black; FONT-FAMILY: Arial" xml:lang="EN-US"><br />
</span><p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; COLOR: black; FONT-FAMILY: Arial" xml:lang="EN-US"><tt><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial; mso-ansi-font-size: 12.0pt; mso-bidi-font-size: 12.0pt" xml:lang="EN-US"><em>The purpose of business risk analysis during software testing is identifying high-risk applications, which must be tested more thoroughly, and to identify error-prone components within specific applications which must be tested more rigorously. The results of the analysis can be used to determine the testing objectives for the application under test during test planning</em></span></tt><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">.</span></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><strong>INTRODUCTION</strong></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The key to performing a successful risk analysis is to formalize the process. An informal approach to risk analysis methods leads an ineffective analysis process. <br/>The first is Judgment and Instinct. This is the most common approach to performing risk assessment during testing. It is an informal approach using the tester's Knowledge, and experience with past projects, to estimate the amount of testing required for the current project. This can be an effective method, but its primary fault the fact that the tester's experience is not readily transferable to other testers. It is a repeatable approach, but is not formally written down for others to use. <br/>Another approach is Identifying and Weighting Risk Attributes. This approach identifies the attributes that cause a risk to occur. The relationship of the attributes to the risk is determined by weighting. The tester uses weighted numerical scores to determine what areas are at the most risks. The scores can be used to determine what application components should be tester more thoroughly than others, and total weighted scores can be used to compare one application to another.</span></p>
<p/><p/><p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/>The thirth approach is The Software Risk Assessment Package. This is an automated approach, which involves purchasing an assessment package from a vendor. The advantage is the ease of use and the ability to do What-If-Analysis with risk characteristics. Automated risk assessment software packages exist which use the second and third approaches above, however, testers can create their own risk assessment questionnaires with MS Word and do the what-if-analysis with MS Excel. This is the approached used here.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><strong>RISK DIMENSIONS</strong></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">There are three identified risk dimensions. They are Project Structure, Project Size, and Experience with Technology. With respect to project structure, the more structured a project the less the risks. Thus, software development projects, which employ some type of project management/ development life cycle approach, should be at less risk. Project size is directly proportional to risk. The larger the project in terms of cost, staff, time, number of functional areas involved, etc., the greater the risk. Technology experience is inversely proportional to project risk. The more experience the development team has with the hardware, the operation system, the database, the Network, and the development language the less the risk. <br/></span><span><br/><strong>THE METHOD</strong></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Suggested risk assessment approach is a five-step risk assessment methodology. The steps are: <br/>1. Ascertain the risk score <br/>2. Create the risk profile <br/>3. Modify the contributing risk characteristics <br/>4. Allocate test resources <br/>5. Create a risk database</span> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span><br/><em><strong>Ascertaining the risk score.</strong></em></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">This involves conducting a risk assessment review with the development team. During the session a risk assessment questionnaire is used to structure the process. Each question is asked of the group and a consensus is reached as to the perceived level of risk.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The questions are closed-end questions with the possible responses of Low (one point), Medium (2 points), High (3 points), and Not Applicable (0 points). The weighted scores can be used to identify error-prone areas of the application and to compare the application with other applications.</span> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span><br/><em><strong>Creating the risk profile.</strong></em></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span><br/>A weighting system is used to compute a score that reflects the importance of each area. Areas, which are twice as important, can be weighted with a value of two (e.g. an area with medium risk (2 points) could be considered three times as important as the other areas and have a final weighted score of 6 points (the weight times the risk points)). A total score is computed for the project, but the individual scores are used to develop the risk profile. Perry is not specific is his description of how to construct the profile. So, I suggest using the following approach.<br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Once the risk data have been collected, create a spreadsheet, which computes the weighted scores. Sort the tabulated scores from the highest to the lowest (a pseudo-frequency distribution of sorts) and perform a Pareto Analysis (the 80/20 rule) to determine what project areas fall into the upper 20% of the distribution. These are the areas, which are at risk the most and they are the areas, which will need to be, tested the most. The results are rather obvious when charted using Kiviat Charts (radar charts).</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The high risk areas standout visually. The charting should be done based on data sorted in ascending order by question number not on data sorted in descending order by highest to lowest. <br/></span><span><br/><em><strong>Modify the risk characteristics.</strong></em></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Perry argues that once the areas at risk have been identified a proactive approach can reduce risk. He suggests that steps be taken to change the development approach or the project structure in order to reduce risk. When these alternatives are not feasible, the process of using the risk information to decide what areas to test becomes even more critical. <br/></span><span><br/><em><strong>Allocate test resources.</strong></em></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Allocating the most test resources to the high-risk areas, allocating less testing resources to medium risk areas and minimal testing resource to low risk testing areas is suggested. A sound strategy is to assure that all of, or as must of as is possible, of the medium to high risk areas are tested with the scope of the allotted testing resources. A possible split could be to commit 80% of the testing resources to medium and high-risk areas, and commit the remaining 20% to low risk areas. This is again applying the 80/20 rules. <br/></span><span><br/><em><strong>Compile a risk assessment database.</strong></em></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span><br/></span><span>A risk assessment database has two important functions. First, it can be used to improve the risk assessment process itself. Second, the data can be used to help management plan and structure development projects. <br/></span><span><br/><strong>THE RISK ASSESSMENT REVIEW SESSION</strong></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The entire test team along with end user representatives should be included in the session. I recommend conducting the session early in the test process. It should last no more than two hours. It should be run formally by the test team manager who facilitates the session. The session should have two objectives. The first objective is to answer each question on the risk assessment questionnaire. The second is to brainstorm and let the participants voice their concerns about the system under development.</span> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span><br/><strong>RISK ASSESSMENT</strong></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">A risk Assessment should be completed for the projects. Should be conducted following the guidelines below. Risk assessment was used to conduct a one-hour assessment of the project's risk factors. The risk assessment session was conducted with one QA person and five XYZ project development team members present. <br/>The raw data were placed in an Excel workbook, and weighted scores were calculated for each question in each of the Test Documents. The data were analyzed using Pareto Analysis to determine the number of questions to consider (i.e the top 20% in terms of risk). The data were displayed through Kiviat charts. The results revealed three major areas of risk. The first is the lack of user documentation about the process being automated. The second is number of interfaces to other systems. The third is the use of new technology to "pioneer" parts of the system. Use risk analysis to determine where testing should be focused. Since it's rarely possible to test every possible aspect of an application, every possible combination of events, every dependency, or everything that could go wrong, risk analysis is appropriate to most software development projects. This requires judgment skills, common sense, and experience.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><strong><u>Considerations can include:</u></strong> <br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- Which functionality is most important to the project's intended purpose? <br/>- Which functionality is most visible to the user? <br/>- Which functionality has the largest safety impact? <br/>- Which functionality has the largest financial impact on users? <br/>- Which aspects of the application are most important to the customer? <br/>- Which aspects of the application can be tested early in the development cycle? <br/>- Which parts of the code are most complex, and thus most subject to errors? <br/>- Which parts of the application were developed in rush or panic mode? <br/>- Which aspects of similar/related previous projects caused problems? <br/>- Which aspects of similar/related previous projects had large maintenance expenses? <br/>- Which parts of the requirements and design is unclear or poorly thought out? <br/>- What do the developers think are the highest-risk aspects of the application? <br/>- What kinds of problems would cause the worst publicity? <br/>- What kinds of problems would cause the most customer service complaints? <br/>- What kinds of tests could easily cover multiple functionalities? <br/>- Which tests will have the best high-risk-coverage to time-required ratio?<br/></span><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><strong><u>GENERAL RISKS LIST</u></strong></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- Complex - anything disproportionately large, intricate or convoluted.<br/>- New - anything that has no history in the product.<br/>- Changed - anything that has been tampered with or "improved".<br/>- Upstream Dependency - anything whose failure will cause cascading failure in the rest of the system.<br/>- Downstream Dependency - anything that is especially sensitive to failures in the rest of the system.<br/>- Critical - anything whose failure could cause substantial damage.<br/>- Precise - anything that must meet its requirements exactly.<br/>- Popular - anything that will be used a lot.<br/>- Strategic - anything that has special importance to your business, such as a feature that sets you apart from the competition.<br/>- Third-party - anything used in the product, but developed outside the project.<br/>- Distributed - anything spread out in time or space, yet whose elements must work together.<br/>- Buggy - anything known to have a lot of problems.<br/>- Recent Failure - anything with a recent history of failure.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><strong><em>André Vondran<br/>SQA Analyst<br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></em></strong></p>Testing Mobile Phone Applications By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316572008-01-09T20:50:12.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span class="text1"><span lang="EN-US" style="FONT-SIZE: 10pt" xml:lang="EN-US">Two months ago, I started learning about the joys and challenges of testing mobile wireless applications. This article is dedicated to the various tips and tricks I've collected along the way that may help you become productive much more quickly.…</span></span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br></br><br></br></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span class="text1"><span lang="EN-US" style="FONT-SIZE: 10pt" xml:lang="EN-US">Two months ago, I started learning about the joys and challenges of testing mobile wireless applications. This article is dedicated to the various tips and tricks I've collected along the way that may help you become productive much more quickly.</span></span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Reduce Setup Time</span></b></span> <span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt"></span></span></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Find ways to reduce the time required to configure the phone, install the software, and learn about the underlying connectivity. For example:</span></span></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Your carrier or handset manufacturer may enable you to download the Internet settings to your phone rather than trying to discover and then manually key in the obscure settings.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Often the software needs to be installed from a Web site. Use text messages to send long Web addresses. Keying a URL can take several minutes and one false move may mean starting again!</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Learn how to use a computer to install the software. Many manufacturers provide free software that will enable you to add and remove software applications relatively painlessly from a computer.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span class="text1"><b><span lang="EN-US" style="FONT-SIZE: 10pt" xml:lang="EN-US">Figure Out Connectivity</span></b></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Mobile connectivity remains a challenge. But remember, a connection relies on at least four elements:</span></span></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><i style="mso-bidi-font-style: normal"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">1. Configuration of the phone</span></i></span> <i style="mso-bidi-font-style: normal"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">2. The service provided by the carrier (and paid for by the user)</span></span> <br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">3. The connectivity between the carrier’s wireless network and the Internet (where gateways can filter, modify, convert, or even block communications for various reasons)</span></span> <br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">4. And the rest of the connection to the Web/application server, which may include more gateways, firewalls, etc.</span></span> <br/></i><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Phones provided by carriers may have the network settings preconfigured. If you have to configure these settings, you will need to learn about Access Point Names (APN), WAP gateway settings, and, possibly, login details for the service provided by the carrier.</span></span> <br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Understand Your Data Plan</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Carriers may offer a range of data services, from very limited access to a small list of approved Web sites (called a walled garden in the industry) to full "Internet" access that may even allow Voice over IP, video streaming, etc. Some carriers provide clear information on which services are available for each price plan; for others, you may have to research what services and Web addresses work reliably.</span></span> <br/><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Check how much you pay for data before embarking on data-intensive applications. I had a monthly data bill that was more than $300—even though I didn't use any of the installed applications on my phone during that time. However, one of the applications polled its server in the background while I was abroad. At $16/MB transferred, it was an expensive lesson to learn!</span></span> <br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Understand the Impact of Transcoders</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Transcoders help make a majority of the Internet more accessible to mobile devices; however, sometimes transcoders have unexpected consequences for the application you want to test. For browser-based Web content, these problems are often visible and therefore relatively easy to discover. However, for custom applications written to run on the mobile devices, transcoders may adversely affect the data communications and cause the application to report errors or even crash without telling you what's going wrong.</span></span> <br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Find Ways to Understand and Simplify Problems</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">I have found diagnostic client software and diagnostic Web servers particularly useful for discovering and debugging issues with transcoders. Both the client and the server are designed to report the information that is sent and received. Find out whether the content is expected to be transcoded, and if so, how. If not, the data sent by one end should be received unchanged at the destination and vice-versa. The diagnostic software recorded all the data and made problems easier to detect.</span></span> <br/><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">If possible, run your diagnostic software on a personal computer with a wireless data modem to debug transcoder or carrier network issues. Generally this is much easier than trying to debug client software on a phone.</span></span> <br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Use Complimentary Tools</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Find complimentary ways to test using Web browsers for Web-based mobile sites. Firefox has numerous free plug-ins that emulate a phone’s Web browser and make manual testing easier. I use the following: WMLBrowser, Web Developer, User Agent Switcher, and Modify Headers.</span></span> <br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Reduce the Number of Combinations</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">As there are thousands of permutations of phones and carriers, pick an exemplary subset of phones to test with. For instance, when testing Java software (written in Java 2 Micro Edition), I test on classes of phones that include: Nokia Series 60 second and third editions; Sony Ericsson’s Java Platform 6, 7, and 8 phones; and BlackBerry models based on the keyboard layout and operating system version.</span></span> <br/><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Pick popular phones and phones with large and small screens and a variety of keyboards, including: T9 (where the alphabet is split across the numeric keys 2 to 9), QWERY, and other unusual keyboard layouts. Over time you may collect "interesting" phones that help expose application flaws. For example, one of my phone's core software has been highly customized by the carrier and has exposed limitations in applications that appear very quickly. By finding and reporting these issues early, the developers were able to revise their application software so it was much more flexible and robust.</span></span> <br/><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Here's a site that details another way to classify your phones based on the operating system and UI: <a href="http://patterns.littlespringsdesign.com/wikka.php?wakka=UsingDeviceHierarchy" target="blank"><span style="FONT-FAMILY: 'Times New Roman'"><font color="#0000FF">Using a Device Hierarchy</font></span></a>.</span></span> <br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Take Good Notes</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Remember to use the recording tools, now! When you find a possible problem during testing, start by recording immediately what you see, as some messages are transitory and clear within a short period. Find ways to record UI issues, on-screen messages, etc. I use:</span></span></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">A good-quality, inexpensive digital camera with a sturdy, close-up stand</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Free software that takes screenshots on certain types of phones, such as <b style="mso-bidi-font-weight: normal">fexplorer</b> (<span style="COLOR: black">for Symbian Series 60 phones</span>), and similar tools from Research-In-Motion (called JavaLoader) or from shareware vendor</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span class="text1"><span lang="EN-US" style="FONT-SIZE: 10pt" xml:lang="EN-US">Now document what relevant events preceded the problem, including things like signal strength, battery level, as well as actions you performed. Try to reproduce the problem on that device. Before considering the bug final, try similar and different devices, different carriers, etc. While this work may take anywhere from minutes to a few hours, the answers may help the developer hone in on the problem much more accurately and quickly. Be practical, and don't forget to ask the developer for help and advice before you expend too much effort.</span></span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><br/><span class="text1"><b><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Practice Makes Perfect</span></b></span> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US"><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">Testing mobile wireless applications effectively takes time and practice, particularly if you are new to the field. I hope you will find some of these tips useful. Get comfortable configuring and using a variety of phones, and you'll soon be much more capable and productive as a tester of mobile wireless applications.</span></span> <br/><br/><span class="text1"><span style="mso-ansi-font-size: 10.0pt; mso-bidi-font-size: 10.0pt">The next lesson is how to automate some of the testing—a great subject with plenty of scope for you to develop your skills and make a big difference in the industry!</span></span><br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span class="text1"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt" xml:lang="EN-US">It is really hard to get hold of handsets and go through all the configurations and manual testing. There is service available on www.deviceanywhere.com which makes this job a lot easier by giving access to so many handsets out there in real networks. Above all they also provide tools for automating tests.</span></b></span></p>
<p/><p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></p>Test Strategy documents make nice blue fire By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316552008-01-09T20:49:15.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Most organizations assume that test strategy should be defined by test managers, tech leads or executives. Perhaps because anything involving the word ’strategy’ is open to interpretation giving it a mystique-laden abstract quality, therefore it is erroneously offered up to the intellectual elite (said tongue-in-cheek) i.e. upper management as a job function.…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Most organizations assume that test strategy should be defined by test managers, tech leads or executives. Perhaps because anything involving the word ’strategy’ is open to interpretation giving it a mystique-laden abstract quality, therefore it is erroneously offered up to the intellectual elite (said tongue-in-cheek) i.e. upper management as a job function.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">The chairman of Greyhound once said: <b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal">‘a strategist’s job is to see the company not as it is but as it can become’</i></b> (JW Teets)</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"> </span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">The truth is, most of the test strategy documents produced by said upper management, tech leads and the like are really just templated after-thoughts that regurgitate and rubber-stamp the currently accepted development process (hereinafter referred to as the status quo).</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">They’re written to appease some organizational bureaucrat or to give the illusion of productivity from an administrative role but ultimately they are neither read, nor referred to nor in any way applied to the organization in a manner that justifies the half-day management salary it cost to format the headings and line up the dots in the table of contents.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Clearly these types of ‘test strategy’ documents miss the whole point of the game.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">So what type of test strategy would openly defy the institutional silver-back templates and provide real value in a fast-paced, roller-coaster web application agile environment? I’m not talking fifty page tome with boldfaced fonts and anchored sub-sections. I’m talking dog-eared worn-from-use concept that’s carried around lovingly by team members and incorporated into the tasks of daily development life.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">What can this organization become and how can we get there?</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">If you think about it, the idea of a strategy is really pretty simple. It’s taught to us from the time we’re old enough to play checkers and we implement it every day whether we’re consciously aware of it or not. What makes a strategy seem complicated are the multiple levels of engagement. To make it less complicated, I break it down into three basic angles of approach:</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 70.8pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">1. Identify risks and form a response<br/>2. Make efficient use of resources<br/>3. Affect change or realize goals</font></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">When it comes to a software development environment the test strategy should be an ongoing endeavor. It redefines itself iteratively throughout the life of the project.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">No wonder no one reads those test strategy documents that PMO’s mandate. They’re useless in the real world because the idea of regenerating a 4 page table of contents and reformatting all those headings in Word every time you iteratively change your strategy is not scalable if your strategy changes every day.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">HINT: your strategy changes every day.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Think about it. In my little model above, the first approach to strategy is to identify and respond to risks (which incidentally is the only reason a testing role exists in any organization). Newsflash: risks don’t line up passively at the beginning of a project and wait around for you to pick them up, hug them and adopt them for your very own. They’re risks because they contain an element of the unknown. You can only identify them as they become visible. In other words, they change over the course of the project and they are about as predictable as a pit viper.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">In case you weren’t paying attention to my first paragraph, what I’m getting at here is that any thinking person can formulate strategy. The problem with creating an over-arching test strategy is that there is an assumption that the strategic part of testing is taken care of and no further thought is given. There is no way a single document can encompass the dynamic nature of strategy. Nor is there any way a single person can control the shifting aspects of strategy for everyone in the organization. The best strategies arise from a shared vision and are implemented and challenged collaboratively by each member of the team.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">I believe test strategy should be an organic coordinated effort that is produced by the testers themselves in concert with developers and designers. Managers obviously strategize too. Part of that strategy should be to give the team members the tools they need to strategize effectively in their daily tasks. This might involve a way to communicate risks to the team, a daily evaluation of resources and a willingness to respond to changing landscapes. Unfortunately there are so many large organizations out there where a testing function is divided into hierarchical levels where you have the strategizers who do nothing but write tests cases and PMO documents and you have test lackeys who go through and check off tasks given to them by the strategizers and they are never to deviate from the given set. Not only is this inefficient but, man, that test lackey job totally sucks. Testers are generally intelligent creative people, and it’s a total waste of resources not to give them the creative freedom and the business knowledge to take a well-thought out strategic approach to their work on a daily basis.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><strong>My advice if you’re a victim of test strategy documents: steal it and burn it.</strong></font></span></p>
<p/><p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></p>Software Testing Methods By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316532008-01-09T20:47:39.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 36pt; TEXT-INDENT: -18pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><a id="Black" name="Black"></a></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: Arial" xml:lang="EN-US"></span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 36pt; TEXT-INDENT: -18pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><a name="Black" id="Black"></a><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Black - Box Testing</font></span></b><span lang="EN-US" xml:lang="EN-US"><br/><font face="Times New Roman">In using this strategy, the tester views the program as a black - box, tester doesn't see the code of the program: Equivalence partitioning, Boundary - value analysis, Error guessing.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 36pt; TEXT-INDENT: -18pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">White - Box Testing</font></span></b><span lang="EN-US" xml:lang="EN-US"><br/><font face="Times New Roman">In using this strategy, the tester examine the internal structure of the program: Statement coverage, Decision coverage, condition coverage, Decision/Condition coverage, Multiple - condition coverage.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 36pt; TEXT-INDENT: -18pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Gray - Box Testing</font></span></b><span lang="EN-US" xml:lang="EN-US"><br/><font face="Times New Roman">In using this strategy Black box testing can be combine with knowledge of database validation, such as SQL for database query and adding/loading data sets to confirm functions, as well as query the database to confirm expected result.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 36pt; TEXT-INDENT: -18pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Test Script</font></span></b><span lang="EN-US" xml:lang="EN-US"><br/><font face="Times New Roman">Type of test file. It is a set of instructions run automatically by a software or hardware test tool.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt 36pt; TEXT-INDENT: -18pt; mso-list: l0 level1 lfo1; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Suite</font></span></b><span lang="EN-US" xml:lang="EN-US"><br/><font face="Times New Roman">A collection of test cases or scripts.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt" xml:lang="EN-US"><font face="Times New Roman"><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></font></span></i></p>Software Life Cycle By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316512008-01-09T20:46:12.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><tt><b style="mso-bidi-font-weight: normal"><i><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">"The full…</span></i></b></tt></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><tt><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-style: italic; mso-bidi-font-family: Arial" xml:lang="EN-US"></span></tt></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><tt><b style="mso-bidi-font-weight: normal"><i><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">"The full business, from initial thinking to final use, is called the product's life cycle."</span></i></b></tt></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Project</span></b> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">A user defined software test effort. Projects contain the specific test plans, test procedures, test cases, defect information, test schedule information, and performance data used to test software applications and track results.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Pre-Alpha</span></b> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Pre-Alpha is the test period during which the product is made available for internal testing by QA, Information Development and other internal users. Shipping Pre-Alpha drops to external customers during this time is explicitly for the purpose of getting feedback about the implementation or usability of one or more features.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="Alpha" id="Alpha"></a></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Alpha</span></b> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Alpha is the test period during which the product is complete and usable in a test environment but not necessarily bug-free. It is the final chance to get verification from customers that the tradeoffs made in the final development stage are coherent.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Entry to Alpha<br/></span></b><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">All features complete/testable (no urgent bugs or QA blockers, including automation) <br/>High bugs on primary platforms fixed/verified <br/>50% of medium bugs on primary platforms fixed/verified<br/>All features tested on primary platforms<br/>Purify run on post-FF drop to obtain baseline<br/>Performance measured/compared to previous release (user<br/>functions)<br/>80% of automation complete (not including UI tests)<br/>Media verified by QA Doc review started <br/>Usability testing and feedback (ongoing) <br/>Alpha sites ready for install <br/>Final product feature set Determined</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="Beta" id="Beta"></a></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Beta <br/></span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Beta is the test period during which the product should be of "FCS quality" (it is complete and usable in a production environment). The purpose of the Beta ship and test period is to test the company's ability to deliver and support the product (and not to test the product itself). Beta also serves as a chance to get a final "vote of confidence" from a few customers to help validate our own belief that the product is now ready for volume shipment to all customers.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Entry to Beta</span></b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">At least 50% positive response from Alpha sites<br/>All customer bugs addressed via patches/drops in Alpha (except OAR bugs)<br/>100% run to plan<br/>Secondary platform/compatibility testing complete: <br/>All bugs fixed/verified <br/>Bug fixes regression/confidence tested<br/>Bug fix rate exceeds find rate consistently for two weeks<br/>Preliminary release notes available<br/>Beta sites ready for install<br/>Second doc review complete</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">GM</span></b> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">(Golden Master) <br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">GM is the test period during which the product should require minimal work, since everything was done prior to Beta. The only planned work should be to revise part numbers and version numbers, prepare documentation for final printing, and sanity testing of the final bits.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Entry to Golden Master</span></b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Beta sites declare the product is ready to ship <br/>All customer bugs addressed via patches/drops in Beta <br/>All negative responses from sites tracked and evaluated <br/>Support declares the product is supportable/ready to ship <br/>QA-qualified</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">U</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">/</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">H</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">bugs re-verified in final drop <br/>All patches delivered to Beta sites included in final drop <br/>Final drop selectively regression tested with no new</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">U</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">/</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">H</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">bugs<br/>Bug find rate is lower than fix rate and steadily decreasing<br/>Docs signed off</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><a name="FCS" id="FCS"></a></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">FCS</span></b> <span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">(First Customer Ship) <b><br/></b></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">FCS is the period which signifies entry into the final phase of a project. At this point, the product is considered wholly complete and ready for purchase and usage by the customers.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Entry to FCS</span></b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Product baked for two weeks with no new urgent bugs (multiple attempts may be required)<br/>Product team declares the product is ready to ship<br/>OHG (Hand-off to QA) approval to ship</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Test Phase</span></b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"><br/></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Pre - determined period of QA evaluation of each software build</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Builds<br/></span></b><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">In many software projects, programming and testing are treated as separate phases. Code units are written and unit tested before any form of integration or system testing. Although this approach may be acceptable on small projects, there are many advantages to overlapping development and testing activities. Builds, which are fundamental approach to testing, allow overlapping development and testing activities.</span></p>
<p/><p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></p>QA Practices and Procedures By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316492008-01-09T20:44:24.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Are you a new in Software Quality Assurance? Please let this be a reference to get you started learning all about…</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><i><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US"></span></i></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Are you a new in Software Quality Assurance? Please let this be a reference to get you started learning all about SQA.</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Testing Methodology</span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The following is an overview of the quality practices of Software Quality Assurance team:</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">-The iterative approach to software development presents a significant challenge for</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">SQA</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">. The iterative, rapid deployment process is characterized by a lack of strict adherence to a traditional waterfall development methodology (marketing first specs the feature set, then engineering refines the marketing requests into more detailed specifications and a schedule, then engineering starts building to specification and SQA starts building tests, then a formal testing cycle, and finally product release). Here is a variant of development:</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">As progress is made toward a release, the first priority features are done to a significant level of completion before much progress is made on the second priority features. A similar approach is taken for the hopefully and third priority features. The first priority feature list is all that has to be completed before a product is feature complete, even though, there has been time built into the schedule to complete the second priority, as well.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Other than the initial OK from the executive team that they want a particular product built, there is not a rigorous set of phases that each feature must pass.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Developers (designers, coders, testers, writers, managers) are expected to interact aggressively and exchange ideas and status.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">By not going heavily into complete specifications, the impact of a better idea along the way need not invalidate a great deal of work.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">One prototype is worth a pound of specification. However, this does not mean that large scale changes should not be specified in writing. Often times, the effort to do paper based design is significantly cheaper than investing in a working prototype. The right balance is sought here.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Complementing the strategy of iterative software development, the SQA testing assessment is accomplished through personal interaction between SQA engineers and Development engineers. Lead SQA engineers meet with the development team to assess the scope of the project, whether new features for an existing product, or the development of a new product.</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Feature</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">, function,</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">GUI</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">, and cross-tool interaction are defined to the level of known attributes. When development documentation is provided, the understanding of the SQA engineer is greatly enhanced. The lead SQA engineer then meets with the test team, to scope the level and complexity of testing required. An estimate of test cases and testing time is arrived at and published, based upon the previous discussions.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Working with the development team, the SQA team takes the builds, from the first functioning integration, and works with the features as they mature, to determine their interaction and the level of testing required to validate the functionality throughout the product.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The SQA engineers, working with existing test plans and development notations on new functionality, as well as their notes on how new features function, develop significant guidelines for actual test cases and strategies to be employed in the testing. The SQA engineers actively seek the input of the development engineers in definition and review of these tests.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Testing is composed of intertwined layers of manual ad hoc and structured testing, supplemented by automated</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">regression</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">testing which is enhanced as the product matures.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Test Plan Components</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Test requirements based on new features or functions.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Specific testing based on the features defined as Development Priority 1. There must be a plan in place for these features and they must be scheduled for testing. A product release date will be slipped in order to complete adequate testing of the Priority 1 features.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Specific testing based on new features or functions defined as Development Priority 2. There must be a plan in place for these features and they must be scheduled for testing. If testing of the Priority 1 features impacts adequate testing of these, they may be dropped from the product.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Specific testing based on new features or functions defined as Development Priority 3. Software Quality Assurance will not schedule or plan for these features. However, Priority 3 completed prior to</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Functional Freeze</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">will be added to the SQA Priority 2 for testing and appropriate risk assessment will be taken with respect their inclusion in the released product.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">SQA has its own set of Priority 1, Priority 2, Priority 3, which include not only the Development activities, but also testing required as due diligence for product verification prior to shipment.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l2 level1 lfo3; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Priority 1</span></b><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">, features include the testing of new features and functions, but also a defined set of base installations, program and data integrity checks, regression testing, documentation (printed,</span> <span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">HTML</span> <span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">and on-line Help) review and final "confidence" (high level manual or automated tests exercising the most frequently used features of the product) checks on all media to be released to the public. Products being distributed over the Web also have their Web download and installation verified.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l2 level1 lfo3; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Priority 2</span></b><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">, include a greater spectrum of installation combinations, boundary checking, advanced test creation and more in-depth "creative"</span> <span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">ad hoc</span> <span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">testing.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; tab-stops: list 0cm"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l2 level1 lfo3; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Priority 3</span></b><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">, usually reflect attempts to bring greater organization to the SQA effort in documentation of test scripts, creation of Flashboards for metric tracking, or expanded load testing.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Testing</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">One of the test methods SQA team practice is "</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Black Box</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">" testing. The SQA engineers, like the customers whom they attempt to emulate, are isolated from the source code and must rely upon their understanding of the application and its features and functions. SQA engineers work with Development engineers toward development of code which lends itself to the exercise of automated test tools, thus providing for stable, repeatable, reliable testing of base features and functions. The deductive, reasoning, creative skills of the SQA engineers are thus freed from the more repetitive tasks to focus on development of more user centric testing, which expands the scope coverage.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Manual Testing</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">GUI</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- SQA team members upon receipt of the Development builds, walk through the</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">GUI</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">and either update existing hard copy of the product Roadmaps, or create new hard copy. This is then passed on to the Tools engineer to automate for new builds and regression testing. Defects are entered into the bugs tracking database, for investigation and resolution. Questions about GUI content are communicated to the Development team for clarification and resolution. The team works to arrive at a GUI appearance and function which is "customer oriented" and appropriate for the platform, Web, UNIX, Windows, Macintosh. Automated GUI regression tests are run against the product at Alpha and Beta "Hand off to QA" ,(</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">HQA</span><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">) to validate that the GUI remains consistent throughout the development process. During the</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Alpha</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">and</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">Beta</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">periods, selected customers validate the customer orientation of the GUI.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Features & Functions</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- SQA test engineers, relying on the team definition, exercise the product <a href="http://www.sqatester.com/glossary/index.htm#feature"><span style="mso-bidi-font-family: Arial"><font color="#330066">features</font></span></a> and functions accordingly. Defects in feature/function capability are entered into the defect tracking system and are communicated to the team. Features are expected to perform as expected and their functionality should be oriented toward ease of use and clarity of objective. Tests are planned around new features and regression tests are exercised to validate existing features and functions are enabled and performing in a manner consistent with prior releases. SQA using the exploratory testing method manually tests and then plans more exhaustive testing and automation. Regression tests are exercised which consist of using developed test cases against the product to validate field input, boundary conditions and so on... Automated tests developed for prior releases are also used for regression testing.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Installation</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- Product is installed on each of the supported operating systems in either default, flat file configuration, or with one of the supported databases. Every operating system and database, supported by the product, is tested, though not in all possible combinations. SQA is committed to executing, during the development life cycle, the combinations most frequently used by the customers. Clean and upgrade installations are the minimum requirements.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Documentation</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- All documentation, which is reviewed by Development prior to Alpha, is reviewed by the SQA team prior to Beta. On-line help and context sensitive Help are considered documentation as well as manuals, HTML documentation and Release Notes. SQA not only verifies technical accuracy, clarity and completeness, they also provide editorial input on consistency, style and typographical errors.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Automated Testing</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">GUI</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- Automated GUI tests are run against the product at Alpha and Beta "Hand off to QA" (HQA) to validate that the GUI has remained consistent within the product throughout the development process. The automated Roadmaps walk through the client tool windows and functions, validating that each is there and that it functions.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Data Driven</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- Data driven scripts developed using the automation tools and auto driver scripts are exercised for both UNIX and Windows platforms to provide repeatable, verifiable actions and results of core functions of the product. Currently these are a subset of all functionality. These are used to validate new builds prior to extensive manual testing, thus assuring both Development and SQA of the robustness of the code.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Future</span></b> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">- Utilization of automated tools will increase as our QA product groups become more proficient at the creation of automated tests. Complete functionality testing is a goal, which will be implemented feature by feature.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="FONT-SIZE: 14pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Reporting</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">The SQA team lead produces a formal test plan which is submitted to the project team for validation and information. Input is solicited for the understanding of requirements, so that the plan can accurately reflect the testing which will be required. Because of the variances between Development teams' style of disseminating feature, function scope and definition, the amount of information available is not always consistent. The SQA lead attempts to establish a good working relationship with the Development team, so that questions can be asked and comprehensive, identical understanding of the product will exist. The lead also pairs a QA engineer with a development counter part to facilitate day to day interaction.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">SQA engineers produce status sheets for distribution to the members of the product teams. The data on the status sheets have included: features, functions, time estimated to test, time consumed and amount of testing yet to be done, the last has proved to be too subjective. Some test engineers include the staff assigned to specific testing and the percentage complete. The later is difficult to estimate due to the iterative process, as feature and function specifics change often and rapidly during the development cycle, as they are refined. The evolving report, which appears to give the most information, is one which lists the features, the build in which they were tested or re-tested and the pass or fail status of the test</span><span lang="EN-US" style="FONT-SIZE: 8pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">SQA provides monitoring of the defects which appear in the product, through the use of QA designed Flashboards (graphical representations of the aggregate numbers of defects) and reports. The defects found in the product are recorded in a Bug Tracking database, where the information is made available to the development group. Information stored in the database then provides statistical and trend analysis for defect find rates and product areas. This information is compared to that presented by the Development team and the Product team. Customer support is kept abreast of these defects and influences the priority assigned to the defect by the team.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">Team leads have established directories for their products in which test plans and weekly status reports have been posted. These are updated weekly by the team lead and reviewed by the manager and linked or posted to the QA home page on the intranet.</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">QA managers work with their teams to assess better forms and methods of information dissemination. These reviewed with the larger engineering and project teams so that the teams feel they understand the scope of work to be done by SQA and the status of a project currently being tested.</span></p>
<p/><p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></p>QA and Lean By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316472008-01-09T20:42:45.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">Recently I’ve had the opportunity to participate in a project that is experimenting with a lean software development environment and so have made a few observations about where QA fits into a <b style="mso-bidi-font-weight: normal">lean</b> methodology.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">Not being an expert…</font></span></p>
<p/><p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">Recently I’ve had the opportunity to participate in a project that is experimenting with a lean software development environment and so have made a few observations about where QA fits into a <b style="mso-bidi-font-weight: normal">lean</b> methodology.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">Not being an expert in lean methodology, it’s possible (and likely) that the process my team is following isn’t strictly lean but some subjective interpretation of it.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">The central idea of lean is that you never do anything until the moment when it is needed and since there is no concept of iterations, schedules tend to unfold organically. Priorities for tasked out work are set daily and decisions about what gets done is made the day it is worked on. Like agile, documentation takes a back seat to conversations about design and requirements.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">This means that when decisions get made, we must rely on the frailty of human memory or a scribbled one-line note on a sticky as documentary evidence.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">I’ve often said that the role of QA is to be the memory for the organization. We spend a lot of time conversing with various stakeholders in various departments.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">We take these conversations and turn them into scenarios that form the basis of our criteria for testing. The result of all of this is that the QA engineer has a more holistic perspective of the end product. Our soul mates in the development process are the business analysts and architects of the system.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">I was excited to get the opportunity to be a part of a lean project. I’ve been reading about it and I like the underlying concepts but mostly I wondered just how it would incorporate QA into the process.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">So far my conclusion is that it doesn’t. When I say “it doesn’t”, I don’t mean this in a “the developers are so good they test all their own work and deliver perfect software” kind of way. I mean, in my opinion the biggest weakness of the lean process we’re following is that in encourages piecemeal delivery to such a degree that those working on the project completely lose sight of the big picture. The code is wonderfully modular, and the bits that get delivered to me for testing work very well in their own right, but in the end they don’t play well together. This team’s implementation of lean doesn’t encourage a collaborative environment but instead forces a handful of people into a role of systemic analysis.</font></span></p>
<p><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US"><font face="Times New Roman">And as the project progresses the team becomes increasingly dependent on the few people who have a systemic understanding of the system to clarify requirements and tell them how it should work.</font></span></p>
<p><font face="Times New Roman"><i style="mso-bidi-font-style: normal"><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">Can the Lean concept work with QA?</span></i> <span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">One of our team members made the observation that the process we were following tended to create a much higher burden on QA because developers had no impetus or opportunity for identifying the interdependencies. A big advantage of Lean is that you get to a mature stage of development extremely quickly and this can be very useful in situations where an organization has been paralyzed in the requirements analysis stage (which was the case with the client we’re working with). The client has been very impressed with short turnaround time. We’ve been able to deliver a working system in about half the time it took our predecessors just to hash through the requirements process.</span></font></p>
<p><font face="Times New Roman"><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">I think for this particular client, it works very well because they know they will change their minds about what they want at any stage of the process. It was less important for them to have a top quality system. That said, we found an unusually high number of major issues post-release. Part of the legacy of our ‘lean’ philosophy was that we had no automated acceptance tests and since the original testers and business analysts are no longer on the project, we lost some of the original understanding (or memory) of how the system was supposed to behave. This problem was exacerbated by a lack of documentation.</span></font></p>
<p><font face="Times New Roman"><span lang="EN-US" style="mso-ansi-language: EN-US" xml:lang="EN-US">Even the stories we developed from were simply keywords with no acceptance criteria and no elaboration as to what those keywords meant. As I tested the requirements, I spent alot of time talking to developers about what they had built, talking to the business about what they had asked for and trying to manage the large gaps between the two.</span></font></p>
<p><font face="Times New Roman">So back to the original question. Can a lean environment support a QA or testing role? I believe it can and I believe the problems with this lean project are rooted in the interpretation of the philosophy. I don’t think lean means: don’t do anything that takes time. And that was the tendency here. I</font></p>
<p><font face="Times New Roman">think the underlying philosophy is fantastic. Basically the idea is to be smart about what you spend your time on and when. In our case, the lean philosophy was more like bare bones. We accomplished a great deal but in the end I believe it was a skeleton of what we needed.</font></p>
<p><font face="Times New Roman">Spending a little more time in analysis and offering the developers more of an opportunity to understand the context of their work could have given us a little more flesh on those bones.</font></p>
<p><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></p>QA and Developers By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316452008-01-09T20:40:53.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What QA needs from Developers?…</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What QA needs from Developers?</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><u><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Information</font></span></u></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What the product is and how it works</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">How the product is intended to be used</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What parts of it are at greater risk of failure</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">The schedule for remaining development</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Status and details of any changes or additions to the product</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Status of reported problems <br/></font><b style="mso-bidi-font-weight: normal"><br/><u><font face="Times New Roman">Services</font></u></b></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Provide timely information to the testers</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Respond quickly to reported problems</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Stay synchronized with the project schedule</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Collaborate to set an appropriate standard of quality</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Involve testers in all decisions that may impact them</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l0 level1 lfo1; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Seek to understand the test process</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What Developers need from QA?</font></span></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><u><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Information</font></span></u></b><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Problems found</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What will be tested</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Schedule for testing</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Status of the current test cycle</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 0cm"><span lang="EN-US" style="FONT-SIZE: 9pt; FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">What information testers need to work effectively <br/></font><b style="mso-bidi-font-weight: normal"><br/><u><font face="Times New Roman">Services</font></u></b><br style="mso-special-character: line-break"/><br style="mso-special-character: line-break"/></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Provide timely information to development</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Seek to understand the general process of creating software</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Seek to understand the product and it's underlying technologies</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Respond quickly to new builds</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Stay synchronized with the schedule, don't delay the project</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt; TEXT-INDENT: 0cm; mso-list: l1 level1 lfo2; tab-stops: list 36.0pt"><span lang="EN-US" style="FONT-FAMILY: Wingdings; mso-fareast-font-family: Wingdings; mso-bidi-font-family: Wingdings" xml:lang="EN-US"><span style="mso-list: Ignore">Ø </span></span><span lang="EN-US" style="FONT-SIZE: 9pt" xml:lang="EN-US"><font face="Times New Roman">Collaborate to set an appropriate standard of quality</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><strong>André Vondran<br/>SQA Analyst</strong><br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></span></p>Non-Functional Testing? By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316432008-01-09T20:39:37.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">There's an interesting discussion going on at the Association for Software…</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i style="mso-bidi-font-style: normal"><u><span lang="EN-US" style="FONT-SIZE: 16pt" xml:lang="EN-US"><font face="Times New Roman"></font></span></u></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">There's an interesting discussion going on at the Association for Software Testing discussion list on non-functional testing. Basically, they think the term is weak and is looking for alternatives. <i>"Non-functional" is a rather bizarre term, though it seems to be gaining acceptance.</i><br/><br/>Yes, that's kind of a weird term. Personally, I have a problem with "requirements" in most commercial organizations. I have worked in organizations where the requirements process is followed to the letter, the documents are created, and someone comes over to the engineering group and asks "How long will this take to build?"<br/><br/>We reply "Well, as written, about two years. But if you take out bullet points C, F, and G, I can do it by myself in about a month." And, magically, there is a new version of the document with C, F, and G removed. If that was the case, could we ever <b>really</b> say that C, F, and G were "Required"? <br/><br/>Non-functional testing has the same problem. It is lingo that doesn't -quite- match up with common sense English. Now, I have given up on trying to correct the requirements term; it's just too entrenched.</font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman">Now, then again eliminating non-functional ... we might have a shot at that.<br/><br/><b style="mso-bidi-font-weight: normal">I like Para-functional.</b></font></span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" xml:lang="EN-US"><font face="Times New Roman"><b style="mso-bidi-font-weight: normal">André Vondran<br/>SQA Analyst<br/><a href="http://www.aqasol.com.ar">www.aqasol.com.ar</a></b></font></span></p>Minimizing Software Defects via Inspections. By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316412008-01-09T20:38:22.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">A major ingredient to reducing development life cycle time is to eliminate defects before they happen. By reducing the number of defects that are found during your quality assurance testing cycle, your team can greatly reduce the time it takes to implement your software project.…</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><b style="mso-bidi-font-weight: normal"><i><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'; mso-bidi-font-family: Arial" xml:lang="EN-US">A major ingredient to reducing development life cycle time is to eliminate defects before they happen. By reducing the number of defects that are found during your quality assurance testing cycle, your team can greatly reduce the time it takes to implement your software project.</span></i></b></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Many of us have experienced projects that drag on much longer than expected and cost more than planned. Most times, this is caused either from inadequate planning (requirement collection and design) or from an inordinate number of defects found during the testing cycle. <br/><br/>The key to reducing software defects is to hold regular inspections that find problems before they occur. Below is a list of 5 Tips for Reducing Software Defects:<br/><br/></span><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US">1. <b>Conduct Requirement Walkthroughs</b></span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">- The best time to stop defects is before coding begins. As the project manager or requirements manager begins collecting the requirements for the software, they should hold meetings with two or more developers to ensure that the requirements are not missing information or are not flawed from a technical perspective. These meetings can bring to surface easier ways to accomplish the requirement and can save countless hours in development if done properly. As a rule of thumb, the requirements should be fully reviewed by the developers before the requirements are signed off.<br/><br/></span><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US">2. <b>Conduct Peer Code Reviews</b> -</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Once coding begins, each programmer should be encouraged to conduct weekly code reviews with their peers. The meeting is relatively informal, where the programmer distributes source code listings to a couple of his/her peers. The peers should inspect the code for logic errors, reusability and conformance to requirements. This process should take no more than an hour and if done properly, will prevent many defects that could arise later in testing. <br/><br/></span><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US">3. <b>Conduct Formal Code Reviews</b> -</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Every few weeks (or before a minor release), the chief architect or technical team leader should do a formal inspection of their team's code. This review is a little more formal, where the leader reviews the source code listings for logic errors, reusability, adherence to requirements, integration with other areas of the system, and documentation. Using a checklist will ensure that all areas of the code are inspected. This process should take no more than a couple of hours for each programmer and should provide specific feedback and ideas for making the code work per the design.<br/><br/></span><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US">4. <b>Document the Results</b> -</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">As inspections are held, someone (referred to as a scribe) should attend the meetings and make detailed notes about each item that is found. Once the meeting is over, the scribe will send the notes to each team member, ensuring that all items are addressed. The scribe can be one of the other programmers, an administrative assistant, or anyone on the team. The defects found should be logged using your defect tracking system and should note what phase of the life cycle the defect was found.<br/><br/></span><span lang="EN-US" style="FONT-FAMILY: Arial" xml:lang="EN-US">5. <b>Collect Metrics</b> -</span> <span lang="EN-US" style="FONT-SIZE: 10pt; FONT-FAMILY: Arial" xml:lang="EN-US">Collect statistics that show how many defects (along with severity and priority) are found in the different stages of the life cycle. The statistics will normally show over time that when more defects are resolved earlier in the life cycle, the length of the project decreases and the quality increases.</span></p>
<p/><p><strong>André Vondran<br/>SQA Analyst<br/></strong><a href="http://www.aqasol.com.ar"><strong>www.aqasol.com.ar</strong></a></p>Is Capacity Planning Still Relevant? By André Vondran - Software Quality Assurance Analysttag:knolstuff.com,2008-01-09:1781665:BlogPost:316392008-01-09T20:37:11.000ZVondran Andrehttps://knolstuff.com/profile/VondranAndre
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">For many years we have studied, debated and improved our knowledge of application and system performance. We know that a mature IT organization is one that measures, plans for the future, then measures again, striving for continuous improvement without painful surprises. Business performance management and truly aligned IT service optimization require capacity planning. We…</span></p>
<p class="MsoNormal" style="MARGIN: 0cm 0cm 0pt"><span lang="EN-US" style="FONT-FAMILY: 'Book Antiqua'" xml:lang="EN-US">For many years we have studied, debated and improved our knowledge of application and system performance. We know that a mature IT organization is one that measures, plans for the future, then measures again, striving for continuous improvement without painful surprises. Business performance management and truly aligned IT service optimization require capacity planning. We know that.<br/><br/>But the continuing trend over the years has been for CIOs to be asked to do more with less. There is constant management pressure to cut hardware, software, facilities, and personnel costs. It is no surprise that vendors are responding to this opportunity by offering operating systems, <span style="BACKGROUND: aqua">virtualization</span> environments, application platforms, and other infrastructure elements that they purport will help organizations cut costs by automating or eliminating tasks such as capacity planning which have traditionally been considered as necessary for systems management. Trade journalists and analyst organizations are eager to join with vendors in promoting this idea that technology can now manage itself. The goal is to reduce the number of system administrators needed per server. Managers want to believe claims that various OS, <span style="BACKGROUND: aqua">virtualization</span>, or middleware elements can decrease administrator-to-server ratios and reduce the cost of managing systems.<br/><br/>With all of these influences conspiring, it seems inevitable that the question will be asked more and more often. Do we still need capacity planning? We know the answer is yes. But how do you convince a manager who does not subscribe to the same common sense principles that we as performance analysts embrace? How do you eloquently and convincingly answer the question?</span><span lang="ES-AR" style="FONT-FAMILY: 'Book Antiqua'; mso-ansi-language: ES-AR" xml:lang="ES-AR"></span></p>
<p/><p/><p><strong>André Vondran<br/>SQA Analyst<br/></strong><a href="http://www.aqasol.com.ar"><strong>www.aqasol.com.ar</strong></a></p>