Code testing, Code Reviews and Quality Assurance

Code reviews
Even if your code is perfect :D code review and tests are prerequisite to get your code merged to the main distribution. It's a good way to spot issues with code early enough, and to make sure it's easy for other developers to read the code and work on it.

Code reviews on Launchpad

QA & Tests
See Quality Assurance for some background information about Quality Assurance. Basically, it's about making sure we maintain high standards of quality and stability when we release new versions of software.

Automated tests
Automated tests help to reduce number of tests that have to be done manually, to spot issues earlier and reduce the quantity of repetitive tasks QA testers have to perform.

A good way to review code is to write automated test for it, like:
 * Unit test (with QUnit for javascript code)
 * Functional test (with Windmill for in browser test)

The author of the code should create most of the tests, especially the unit test ones - it helps the code reviewer to write some of the tests to understand the code, but it's still the job of the developer.

Unit tests
The code reviewer is there to help and to show examples, but as the author of the code, you're ultimately responsible to ensure 100% coverage of the code you produce. It's better if the code is directly written in a way that makes it easier to write the tests - and to have the same person writing the tests and the code is the best way to make sure of that. : ) The author of the code could even begin to write unittest "first", before writing any code: Test-driven development.

Unit tests are helpful to evaluate code quality, the number of tests and mockup you have to write to cover all the code path of a function gives a good mesure of its code complexity.

When code is being reviewed:
 * If there is no unittest: the reviewer can help to write unittest code afterwards, it haa to read the code and figure what it is supposed to do, and capture that in a test
 * If there is unittests: the reviewer can read and run unittest code, in order to known what is code is supposed to do, and then read corresponding code in context.

Functional browser tests
Automated functional browser test like provided by windmill helps to capture more general use cases (rather than API) and to ensure they are still working after code updates.

In the future, QA testers will manage automated functional tests - they will still be produced by the developer, but the QA tester will be there to make sure they cover all functionalities and try to automate his own work as much as possible (the only things left to do manually are when it's too difficult to automate, or when it becomes more subjective).

Use cases included in the specifications provide a good basis for functional testing (automated or not) as they describe a possible path that can be tested. A nice way to help writing functional tests is to define manual "discrete" steps (i.e browser interaction) needed for validating the core fonction of the site, like:


 * Open firefox
 * Give focus to the location bar
 * Type http://www.ortographics.com/_webdev/hackit/gt/u/www.google.ca/
 * Hit Enter
 * Check that "Welcome Guest" is displayed
 * Check that "Hack It!" button is displayed
 * Over "Hack It!" button
 * Check that "this is the tooltip for the hackit button" is displayed
 * Check that "Options" link is displayed
 * Click "Options" link
 * Check that drop down menu is displayed with an animation
 * Click "Options" link again
 * Check that drop down menu is hidden with an animation

This would be derived from the "Home page" specifications and mockup:
 * A welcome message should be displayed
 * Buttons should have tooltip
 * Drop down menu should be animated

It can then be translated to an automated functional test: for example windmill provide a "recorder" mode that record user interaction through the browser, and offer to save and replay it later. This can be further automated with a windmill script, that check the dom for tag, title= attribute, simulate click, and check that div are displayed or hidden. Alternatively,

There are even "Behaviour Driven Development" tool, that allows to translate "Plain text" behaviour descriptions, into automated test code. See:
 * http://cukes.info/
 * http://en.wikipedia.org/wiki/Behavior_Driven_Development

Human testing
However, automated testing (unit tests and functional tests) don't completely replace the need for human testing. It makes it less repetitive (functional tests are especially useful to automate regression tests), and allows to spot some issues earlier in the development process (unit tests), but we'll still need QA testers. (In the previous example, asserting, from the automated test, that the menu are indeed animated properly would be trickier.)

Not an army of QA testers like we usually have in video game studios, but probably at least one Senior QA tester for each game, to coordinate the testing efforts, write testing plans&automated browser tests, and make sure that the key features have been properly tested before putting out a new release. It will be critical to ensure patches quality, especially if we want to have a fast-paced release cycle (every week?).