Minimize Diagnostics in Common Lisp

Diagnostics generated by compilers are quite useful for programmers to spot bugs or inefficiencies in the first place. It is quite often to observe that command options -Wall -pedantic (or even -Werror) are used to invoke gcc. When investigating the situation in Common Lisp, a newcomer would be surprised to see that diagnostics and ways to handle them are actually standardized (just like other implementation-like-stuff-in-other-languages, say disassemble). In addition to the already standardized diagnostics like warning and style warning (the latter is actually a subtype of the former), some CL implementations (notably CMUCL and SBCL) provides efficiency notes, which alerts the user that the compiler has chosen a rather inefficient implementation for some operations. For SBCL users, it is highly recommended to read one part of SBCL manual to learn how to interpret SBCL diagnostics.

In typical development cycle, after writing/modifying some code and compilation, if compiler complains with such diagnostics, you know that somewhere needs your attention. However, you can only sense these new diagnostics if the number of existing diagnostics is quite small or zero. Otherwise you many not even notice the new diagnostics since you already have a bunch of them. In this sense, existing diagnostics are like broken windows, and they discourage you to identify new diagnostics. Your program will deteriorate if you do not fix these broken windows. In summary, to make the diagnostics useful, you have to minimize, or better yet, eliminate existing diagnostics.

There are two ways to stop the compiler emitting diagnostics: removal and muffling. We will discuss them separately. Note that we will use SBCL as our CL implementation in the following discussion.

1. Removing Diagnostics

This is of course our first choice, since diagnostics are indications of some risks in the code. In the following, we give some simple examples to illustrate how to delete such annoying (but useful somehow) complaints.

1.1 Warnings

Typically warnings are serious issues to be resolved immediately. One exception is that sometimes if one file contains multiple style warnings, a warning will be issued for the file itself as well. Therefore to remove such warning requires eliminating related style warnings, as discussed below.

1.2 Style Warnings

1.2.1 Variable X defined but never used.

The first consideration here should be to remove the variable causing diagnostics if possible, which has the additional benefit of simplifying code. However there are cases when we cannot change the function interface. For example, when we write reader macros with set-dispatch-macro-character, the 3rd argument is itself a function with 3 arguments like (stream char1 char2), and typically we do not use either char1 or char2. In this situation, we can use declarations like (declare (ignore char1 char2)). Sometimes, if the argument is used in some scenarios while not in other cases (typically in macro definitions), we can use declaration ignorable. An example here is the anaphoric macro acond2 defined in On Lisp, a declaration (declare (ignorable it)) is needed for the ubiquitous it.

1.2.2 Redefining FOO in BAR.

Typically such style warnings are issued when one file is reload, hence can be simply ignored. However there are some more complicated cases. One example is again related to the reader macro. If one want to use the reader macro in the same file, the definition of the reader macro should be encompassed with (eval-when (:compile-toplevel :load-toplevel :execute), as illustrated in HyperSpec. The problem is that if you write a function instead, every time you load the file, this style warning pops up. The solution? Either use anonymous functions as illustrated in HyperSpec, or spin out related part into a separate file without the eval-when stuff.

1.3 Efficiency Notes

It should be noted that efficiency notes are emitted only when we turn on optimization declarations, e.g. (declare (optimize (speed 3) (safety 0))). It is well known that premature optimization is the root of all evil. Therefore before battling against those notes, one should check whether the function in question is the bottleneck or not. If not, one should remove those optimization declarations, which can make the core simpler and more maintanable; otherwise (when the optimization is really necessary), one can proceed further with techniques discussed below.

As indicated by the informative CMUCL manual on efficiency notes, the solution to dismiss these notes is to provide sufficient type declarations. It should be noted that current SBCL is smart enough to perform type inference, therefore it is unnecessary to clutter the code with type declarations for every expressions.

1.3.1 Doing X to pointer coercion

Such notes are typically emitted for function return values which are of boxed types. For example, on 64-bit machines, double-float is still boxed (it requires exactly 64-bits!), therefore compiling the following functions will get a note doing float to pointer coercion (cost 13) to "<return value>".

Code-list-1

(defun df+ (x y)
  (declare (optimize (speed 3) (safety 0))
           (double-float x y))
  (+ x y))

How to remove these notes? One way is to use unboxed types. For example, in above example, one can use single-float instead of double-float for 64-bit CPUs. However such solution is not always available: one reason is that there are only a few unboxed types, and the other one is that sometimes the boxed types are what we really need: e.g. double-float is required from accuracy point of view. Another solution is to use local functions. Since compiler know how to utilize the return value, the efficiency notes will not be emitted. One example is shown below:

Code-list-2

(defun df-outer ()
  (flet ((df+ (x y)
           (declare (optimize (speed 3) (safety 0))
                    (double-float x y))
           (+ x y)))
    (coerce (df+ 1.0d0 2.0d0) 'float)))

Note that the above example is simply contrived to illustrate that using local functions can remove efficiency notes.

2. Muffling Diagnostics

There are many occasions that above solutions cannot be applied. In this case, we can muffle the diagnostics. The intention of muffling is not that we are going to adopt an ostrich policy for those diagnostics. The underlying rationale is actually as following:

• We acknowledge that there is no way to eliminate the diagnostics.
• The diagnostics do not impose any risks to the program per se.
• The diagnostics have to be suppressed in order not to obscure other diagnostics.

One example is the above code-list-1. If such a standalone function optimized for double-float is really what we want, we cannot remove the efficiency note on 64-bit machines (when will 128-bit CPUs become mainstream?). Since there is no problem with the code, we can safely muffle the annoying note to avoid it to distract our attention further.

Common Lisp does provide a standard way to muffle standard diagnostics (i.e. warnings and style warnings, but not notes since they are not standardized). This is function muffle-warning, however its usage seems not straightforward. SBCL wraps it up and provides a pair of declarations sb-ext:muffle-conditions and sb-ext:unmuffle-conditions to muffle diagnostics. We will use them in the following discussion. If portability is desirable, one should use #+sbcl; however in the following examples, we do not use it for simplicity.

2.1 Local Control

It is preferred that we muffle the diagnostics in a local definition if possible. We can specify which types of diagnostics to muffle and use the pair of extensions for advanced purposes, as illustrated in SBCL manual. Following is an example to illustrate how it is applied to our example (code-list-1) above.

Code-list-3

(defun df+ (x y)
  (declare (optimize (speed 3) (safety 0))
           (double-float x y)
           (sb-ext:muffle-conditions sb-ext:compiler-note))
  (+ x y))

Note the usage of sb-ext:muffle-conditions in line 4 above.

2.2 Global Control

One can muffle diagnostics globally in the way like (declaim (sb-ext:muffle-conditions sb-ext:compiler-note)). However it is not recommend to do so since we will lose the whole point of using diagnostics. Nevertheless using pairs of global declarations is useful sometimes, since it can muffle the diagnostics issued for top level structures, and allow the compiler to complain again if it meets issues in other places.

Let's use code in On Lisp again as another example. When emulating Scheme-like continuations in Common Lisp, Paul Graham defined parameter *cont* as (setq *cont* #'identity) in top level. SBCL emits a warning undefined variable: *cont* when compiling the code. It should be noted that we cannot use defvar for this parameter, as discussed in the text. To muffle the warning, we can add a pair of declarations as in code-list-4 below.

Code-list-4

(declaim (sb-ext:muffle-conditions warning))
(setq *cont* #'identity)
(declaim (sb-ext:unmuffle-conditions warning))

Update: as pointed out by Lars Rune Nøstdal in the comment below, a cleaner way is to use locally. The above example can be simplified as following:

Code-list-5

(locally (declare (sb-ext:muffle-conditions warning))
  (setq *cont* #'identity))

Conclusion

To make the best of compiler diagnostics, it is good to remove the existing ones. Happy hacking beautiful code!