Beta Testing

By Richard Lowe
Posted Wednesday, December 1, 2004

I'm sure as you've cruised around the web you've run across the term "beta release" or "beta test version" or something to that effect. You may also have heard "alpha test" or "public preview" and other similar terms. What the heck do these terms mean?

All well managed product development projects are split into multiple phases, each distinct and each with it's own goals. Generally you begin with a proof of concept, request for proposal, short analysis or things like this. You follow this short cycle with a longer analysis, then a design specification, and finally you implement the program. This consists of the actual coding of the programs and might include documentation as well.

Now you've got a finished project, and what you are supposed to do next is a quality assurance step. Generally, you want to test and test and test until the program (or application system) exactly meets the design specification (which should be treated as a sacred document) and associated standards manuals. It is critical to note that testing is done against the specification and standards manual and nothing else at this phase of the project.

When and only when you complete this testing and the product 100% does what it was designed to do, then you create what is called an alpha test. This consists of sending the code to a select group of users who will pound on the product. Their job is to ensure that the product works in the field under conditions outside of your lab.

Once your product has been tested by some alpha testers, you may want to take some time to correct any issues, then perhaps send it out to the alpha testers again. You might repeat this step one or two times until the number of bugs found is significantly reduced.

Your next step is to widen the group of testers greatly. This is called a beta test, and in it's purpose is to ensure your product works on an even wider base of computers. The theory is you cannot duplicate all conditions in your lab, and the alpha testers are purposely kept to a small, very manageable group.

Beta testers are generally just a number of users who, in exchange for a reduced fee or a free copy of the software, agree to install the program (or system) and use them. They are to report any errors, and understand that the program may have bugs, can crash and might corrupt data.

A program or system might go through one, two or even three rounds of beta testing. The developers would be in constant communication with the testers so as to produce the best possible product.

This is the way it worked, and this is the method the better product managers still employ. I've been a beta tester many times, most seriously a dozen years ago with operating systems such as RSX, RT11, RSTS/E and OpenVMS. One of the major reasons why the Digital Equipment Corporation operating systems were so good (and in many ways are still unsurpassed) - they really knew how to run a solid series of beta tests.

The original concept was very simple, but large companies such as Microsoft have corrupted it until now Beta testing is not really testing. You see, Microsoft has changed Beta testing to really be a "pre-release" or "public preview". What does this mean? It means Microsoft has the option to deliver unfinished or inadequately tested code to tens of thousands of users.

Microsoft does not tend to go through a lot of trouble to get feedback from this vast army of beta testers. Oh yes, they do surveys and send forms, but to do a real beta test you need to be in firm control and be sure your testers are doing something useful.

You see, beta testing is supposed to be part of the development process, not the marketing and promotional process. And, truthfully, that's one of the primary reasons why Microsoft (and other large companies) have had such poor quality assurance in the past few years.

So, basically, beta testing helps ensure that a product is more reliable and works in environments outside of the lab. And that's all it is supposed to do.

About the Author
Richard Lowe Jr. is the webmaster of Internet Tips And Secrets at (http://www.internet-tips.net) - Visit our website any time to read over 1,000 complete FREE articles about how to improve your internet profits, enjoyment and knowledge.