Quality: The Process, Part II [Development environment]
[continued from Quality: The Process, Part II]
Development environment
The best place to start a discussion of practical testing is with the development environment and the tools that you are already using. Rather than a comprehensive discussion of programming practices, which would be a book by itself, I will concentrate on using these tools to facilitate the testing work.
Generally, a development environment for a compiled language consists of a text editor, a compiler, and a linker, plus other tools useful during development, and often these are all part of a single IDE (Integrated Development Environment). This compiled development environment is assumed for this discussion, though there are analogous approaches in other environments.
The first step in producing quality code is to understand your development environment. Although there seems to be a growing trend towards hiding or automating functionality, it is nevertheless important to know what the make or project files are doing, and what options are available and being used. You need to how things work in the case that, "It Just Works," fails.
Assuming you understand how the development environment works, you can begin actually programming. Once a certain amount of code is written, you try to build (i.e., compile and link) the executable. This is, in fact, the most basic form of glass box testing. If there are problems in the source code, or the build environment is not correct, then warnings or errors will be generated.
To make the best use of this functionality, modify the settings in the make or project files to set the highest warning level available. This may produce lots of extras warnings on some projects, but compiler warnings exist for good reasons. A warning almost always indicates that there is an immediate problem or, at least, a lack of clarity within the code that reduces the ease of maintenance and could cause future problems.
Many compilers include an option to treat any warnings as errors, and I recommend enabling this option. Warnings should never be ignored, and this prevents that from happening. Instead, source code should be corrected to eliminate warnings. This may seem like obvious advice to some readers, but my experience working with code from other programmers shows that many programmers routinely ignore warning messages during compilation, a dangerous practice that is contrary to quality development.
Taking this checking one step further, build the program frequently. This allows you to catch the warnings as they are introduced, rather than having a collection of problems at the end of a large coding session. Some warnings indicate problems that may need to be addressed by technical design changes, and it is good to find these problems early. Personally, I rarely write more than a short function between builds.
Black box testing should also be used in the early stages of development, even when features are incomplete. Running the executable regularly helps make sure that errors are not introduced or, when they are, catches them at an early stage. For incomplete features, you can hardcode values for testing, or just assure that the program behaves as expected, considering the missing code.
[continued in Expanding our repertoire]
Development environment
The best place to start a discussion of practical testing is with the development environment and the tools that you are already using. Rather than a comprehensive discussion of programming practices, which would be a book by itself, I will concentrate on using these tools to facilitate the testing work.
Generally, a development environment for a compiled language consists of a text editor, a compiler, and a linker, plus other tools useful during development, and often these are all part of a single IDE (Integrated Development Environment). This compiled development environment is assumed for this discussion, though there are analogous approaches in other environments.
The first step in producing quality code is to understand your development environment. Although there seems to be a growing trend towards hiding or automating functionality, it is nevertheless important to know what the make or project files are doing, and what options are available and being used. You need to how things work in the case that, "It Just Works," fails.
Assuming you understand how the development environment works, you can begin actually programming. Once a certain amount of code is written, you try to build (i.e., compile and link) the executable. This is, in fact, the most basic form of glass box testing. If there are problems in the source code, or the build environment is not correct, then warnings or errors will be generated.
To make the best use of this functionality, modify the settings in the make or project files to set the highest warning level available. This may produce lots of extras warnings on some projects, but compiler warnings exist for good reasons. A warning almost always indicates that there is an immediate problem or, at least, a lack of clarity within the code that reduces the ease of maintenance and could cause future problems.
Many compilers include an option to treat any warnings as errors, and I recommend enabling this option. Warnings should never be ignored, and this prevents that from happening. Instead, source code should be corrected to eliminate warnings. This may seem like obvious advice to some readers, but my experience working with code from other programmers shows that many programmers routinely ignore warning messages during compilation, a dangerous practice that is contrary to quality development.
Taking this checking one step further, build the program frequently. This allows you to catch the warnings as they are introduced, rather than having a collection of problems at the end of a large coding session. Some warnings indicate problems that may need to be addressed by technical design changes, and it is good to find these problems early. Personally, I rarely write more than a short function between builds.
Black box testing should also be used in the early stages of development, even when features are incomplete. Running the executable regularly helps make sure that errors are not introduced or, when they are, catches them at an early stage. For incomplete features, you can hardcode values for testing, or just assure that the program behaves as expected, considering the missing code.
[continued in Expanding our repertoire]
posted by Gregg Seelhoff | 12:41 AM
4 Comments:
I have to disagree about ensuring that all code compiles without warnings.
A warning indicates a potential bug in your code. For example, storing a pointer in an int. For some cases, this is the only option (e.g. SetWindowLongPtr for Win32, which is a macro for SetWindowLong).
How can you get rid of the warning? To cast the pointer to an int.
What does this achieve? It buries the potential bug and makes it very difficult to find later on. But the potential bug is most definitely still there, and now it's there by design instead of by side-effect.
I really don't recommend hiding bugs, by striving to erase every warning.
>I have to disagree about ensuring that all code compiles without warnings.
I disagree with your disagreement. :) Although there may be an exception to the rule, I do not think that you have one here.
>A warning indicates a potential bug in your code.
Exactly. However, it is only a "potential" bug to a machine. To a programmer, it either is a bug or it is not.
If the warning indicates a bug, then fix the bug. If it flags a desired behavior, then clarify the intent, removing the warning in the process.
>I really don't recommend hiding bugs, by striving to erase every warning.
I am not pushing for mindless elimination of warnings; I am advocating exactly the opposite. For every warning, a programmer should examine the issue in question and take an action.
In your example, storing a pointer with SetWindowLongPtr(), one must cast the pointer to LONG_PTR, not int. Specifically, using reinterpret_cast<LONG_PTR>() is exactly the correct solution; it resolves resolves the warning and makes the intent explicit.
An alternate, but less desireable, solution is to eliminate the specific warning by using pragmas around the problematic line of code. This clearly both documents the fact that the line normally produces an error and indicates that the programmer is unclear why or how to resolve it.
The problem with allowing any warnings in the compiler output is that they become routine and, hence, routinely ignored. This does just the opposite of encouraging quality code development.
Hi Gregg, thanks for your reply.
I didn't mean to imply that you would like all warnings to be silenced without thinking about their cause, just that sometimes it is difficult to get rid of a warning without causing a subtle new bug when compiling for another platform.
Regarding the SetWindowLongPtr case, I always do a reinterpret_cast to LONG_PTR, which unfortunately still causes a warning, at least under VC++ Express 2005 and Win32. I would imagine that other compilers which don't report 64-bit portability issues wouldn't issue a warning.
I'm sure it could be #ifdef'd or #pragma'd around though.
Now I understand what you were saying. Essentially, Microsoft committed an error by defining SetWindowLongPtr() as a macro for SetWindowLong(), even though they take different arguments. (Microsoft are not the paragon of quality header files. :))
>I would imagine that other compilers which don't report 64-bit portability issues wouldn't issue a warning.
You are correct that the warning only appears when 'Detect 64-bit Portability Issues' (/Wp64) is selected, at least in the full edition of Visual Studio 2005. (Ironically, casting everything back to LONG also works, but is both wrong and ugly.)
>I'm sure it could be #ifdef'd or #pragma'd around though.
The pragma solution works quite well for this situation, as it very definitely highlights the issue in the code:
#pragma warning ( disable: 4244 )
SetWindowLongPtr ( window, GWLP_USERDATA, reinterpret_cast<LONG_PTR>( data ) );
#pragma warning ( default: 4244 )
If this is a problem in more than a couple places, then banish this error (not of your making) to exile in a separate file/library by creating your own version of the routine, with the workaround defined there, rather than in multiple places within the main code base.
In any event, eliminating the warning helps highlight problems that may arise in the future. I hope this helps.
Post a Comment
<< Home