Fail Better: Turning Software Errors into Documentation

The open-source community has done a great job of promoting good documentation as a necessary trait for the success of a software project, whether it be a utility library, a larger framework or a standalone program.

The Power of Context

I’m going to use Pragma, one of my open source projects, as an example for the rest of this article.

Identify Slippery Slopes

The first step towards improving errors in a software project is to identify as many slippery slopes as we can. I define slippery slopes (also sometimes called “gotchas”) as the areas in a software where users are most likely to make a mistake, due to their complexity or counterintuitiveness.

  • Ask your users. If you already have a user base, they are your best source of knowledge about your software. Look at recurring questions and complaints, as well as the most common “Not an issue” bug reports. What is it that users find it hardest to understand about your project?

Problem-Cause-Solution

Once you have the proper error detection technique in place, you need to write the actual error that will be displayed to your users. While improving the error messages for Pragma, I came up with a pattern to follow for good error messages, which I call Problem-Cause-Solution (P-C-S):

  • Cause: this is the possible cause of the problem. Why is that value null? Did your user forget to set it somewhere? There could be many potential causes, and that’s absolutely fine: you don’t need to identify the exact issue, just provide context to help users debug it.
  • Solution: how can the user solve the problem? Again, there could be many potential solutions. For instance, you might tell users to either adhere to your project’s conventions or use explicit configuration. You can get very smart here and use user input to provide them with code they can simply copy-paste to fix the error.

Applying P-C-S to Pragma

One of Pragma’s slippery slopes was its reliance on convention over configuration: if classes are not defined in predefined locations, the framework cannot automatically perform certain tasks (like finding a record in your database). Users have the option to specify the location manually, but this is buried deep in the API reference since it’s not a common use case.

NoMethodError: undefined method `new' for nil:NilClass
You are attempting to use the Model macro, but no `model.class` skill is defined.This can happen when a required class (e.g. the contract class) is not in the expected location.To solve the problem, move the class to its expected location or, if you want to provide the class manually, add the following to your operation:    self['model.class'] = MyCustomClassIf you don't have the class, you will have to skip the macro that requires it.

Over to You!

Do you have a standard for detecting and documenting errors in your project? Are there any examples of good error handling that you have seen and use as inspiration? Leave a response to this story by hitting the comment icon!

Technology leader and strategic advisor. I work at the intersection of people, software and words.