In this article, we explore how we can provide a user guide for software projects that grow beyond their original scope so that a 1-page readme file may not be sufficient anymore. We will look at a package called Sphinx that helps us write a user guide by adding searching, navigation, and a table of contents to our documentation. Furthermore, we will look at a markup language called reStructuredText, which is much more consistent than Markdown and the preferred choice in Sphinx.
By the end of this article, you have all the tools you need to quickly churn out a user guide, and similar to Markdown, this article is likely all you need to get started with reStructuredText and Sphinx. There is much more to both of them but we cover the basics here that you likely use on a day-to-day basis. We apply this knowledge to our mesh reading library, as the 1-page readme file got rather long. We split it up into different parts and will exemplify how we can turn this rather long readme file into a more navigatable user guide.
Download Resources
All developed code and resources in this article are available for download. If you are encountering issues running any of the scripts, please refer to the instructions for running scripts downloaded from this website.
- Download: meshReadingLib-Part3.zip
In This Series
- Part 1: Documenting code; why bother and how to do it right
- Part 2: How to write a 1-page readme file to document your code
- Part 3: How to write a user guide to document your CFD code
- Part 4: How to annotate C++ code for automatic code documentation
In this article
The need for a user guide
Ok, so let’s recap. We looked in our opening article on code documentation to see the available types: the 1-page readme file, the user guide, and the annotated code documentation. We looked into how to write the 1-page readme file in the previous article and the focus of the current article is to look at the user guide.
As an astute reader of cfd.university, you will have noticed in the previous article that the 1-page readme file for the mesh reading library was rather long. And this is, essentially, where the user guide comes in. Typically, you start your project with a 1-page readme file. This will contain the background and what the software does, the installation process, along with some use cases.
Once your software grows, your use cases will increase and you will end up with a rather long readme file. This is usually the point where you want to grab the file, split it into several sections and package that as a user guide instead. This allows you to be a bit more detailed in various sections and add, for example, a quick start guide, a tutorial section, a contribution section, and so on.
So a user guide is typically only required once your software has grown to a certain size. Of course, if you are planning to write a large piece of software from the start, potentially with others, it is a good idea to think about documentation from the start and build it into your code, not bolt it on at the end.
How to get over a breakup (with Markdown)
In the previous article on how to write the 1-page readme file, I advocated that we need some form of markup language that allows us to write our documentation in clear text. This means that we leave some markup hints in the text and use an appropriate markup language to transform our text into a beautiful HTML/PDF/LaTeX output.
We then looked at the defacto most popular markup language: Markdown. We covered the syntax and wrote a 1-page readme file for both our linear algebra solver and mesh reader library. Job done, right? Let’s just use Markdown and move on. Any documentation that needs to be written will be done with Markdown.
Well, let’s recap some of the issues with Markdown as well. We saw that the inventor of Markdown didn’t care about formalising the markup language into a specification. So others did for him, and we ended up with different Markdown specifications, which all extended the set of very limited instructions to include some additional features such as displaying equations. So, if you want to use Markdown, make sure you use the correct specification for your target platform.
If you have ever used LaTeX to write larger documents, you will appreciate the following comparison: If all you want to do is to write a few pages and format them, use Microsoft Word. It gets the job done and allows you visually to alter the document to your liking. If, however, you have to write a longer piece of work, something which requires figures, tables, equations, a table of contents, etc., then, if you have used LaTeX, you will probably agree that this is the right tool for the job, and you should stay away from Microsoft Word.
Markdown is a bit like Microsoft Word, it is great for writing short pieces of documentation, but if you are planning to write anything more than a page, you may get into trouble, quickly. This doesn’t mean that you can’t use the markup language here, it just means that there may be alternatives available that are better suited for the job, like LaTeX for longer text documents.
The alternative, in this case, is reStructuredText, and we will be looking at it in the next section in more detail.
Your new fling: reStructuredText
Just like Markdown, reStructuredText aims to provide some markup in plaintext that can be used to create formatted HTML/pdf/LaTeX output. It is based on StructuredText with elements of Setext (Structure-enhanced text), both of which had their respective problems. These issues were addressed by reStructuredText, which can be seen as a revised, reworked, and reinterpreted version of StructuredText, hence the name reStructuredText.
reStructuredText was developed as part of the Docutils project, which is a text-processing framework. Think back to how we used Markdown: we provided markup in a plaintext document but then relied on a renderer to give us an output which can be viewed as an HTML document. If the renderer was using a different specification than you for writing the document, then chances are these are not translated correctly to HTML.
To get around this project, reStructuredText is not a standalone solution. It is tightly integrated with the Docutils project which in turn is responsible for creating the desired output (e.g. HTML) from the reStructuredText document. This avoids dependence on the renderer and ensures correct translation from marked-up plaintext documents to HTML.
You see, this makes reStructuredText, in my opinion, a much better markup language than Markdown. However, Markdown still dominates the code documentation space which is likely explained by the somewhat easier-to-remember syntax of Markdown.
So where does that leave us? I would argue that we need to be able to write code documentation in either Markdown or reStructuredText. You’ll likely pick your favourite and then stick with it. It is also possible that your development environment dictates which of these to use. For example, anything related to Python will likely use reStructuredText over Markdown. In the meantime, familiarise yourself with both of them and then pick the one which is best suited for your needs.
We will be writing our user guide in this article using reStructuredText, as there are packages available that provide some decent support to turn our text into an easy-to-navigate user guide, complete with a table of contents, navigation aids, and search functionality. We’ll review the basic syntax of reStructuredText below, and you will likely not need more in your day-to-day writing activities, though it is more powerful than what we review here.
The official documentation for reStructuredText does come with a primer that exposes the basic functionality, which you may want to read alongside this article. It also features a complete list of available commands that you may want to consult if you need to achieve something specific. With that out of the way, let’s look at some of the basic syntax.
Headings
Headings are created by placing an underline under the section/sub-section title. You can use any non-alphanumeric value consisting of any of the following symbols: = - ` : ' " ~ ^ _ * + # < >. The underline has to use the same character and needs to be at least as long as the section title itself (but it can be longer). Using the same underline character for a different section means that both these sections appear at the same level.
1 Main title
============
Headings are indicated by non-alphanumeric characters used as underlines under the section title. These can be any of the following characters: = - ` : ' " ~ ^ _ * + # < >
They must be at least as long as the title itself but typically is just as long as the section title itself.
1.1 Section heading
*******************
If you want to have a sub-section within a section, use a different underline character.
1.1.1 Sub-section heading
-------------------------
You can have even more sub-sections with a different character.
1.2 Section heading 2
*********************
Using the same underline character as used before by a different section title indicates that the current section is at the same level.Paragraphs
Paragraphs are separated by simple line breaks (leave one empty line between paragraphs). There are a few things we can do with paragraphs, from simple bold and italic highlighting to substitutions. The following paragraph example highlights some of the features that are most useful and commonly used.
We can highlight text in **bold** or *italics*. We can also use superscripts such as H :sub:`2` O and superscripts such as E=mc :sup:`2`. Notice that this produces a whitespace between the sub- and superscripts. To avoid the additional whitespace, we have to escape it with the backslash as in H\ :sub:`2`\ O and E=mc\ :sup:`2`
reStructuredTextr also comes with a substitution mechanism so that we can provide an easy-to-remember alias that will be substituted with the actual reStrcutredText syntax. We use vertical bars to indicate text that should be replaced as in |H2O| and |E=mc2| and then provide definitions for them somewhere in the document (it doesn't matter if it is at the start or end of the document).
A new paragraph is created by simply adding an empty line between two paragraphs. You can also add a separating horizontal line if you want using four dashes:
----
You may want to quote some text in your documentation. You can insert footnotes using the [#footnote]_ syntax. You can then define this footnote at the end of your document. Using a leading # sign ensures that footnotes are automatically numbered. If you want to highlight a paragraph visually, for example, to show a quote from an important person, use an indented paragraph. To show a source, leave an empty line between the block quote and the source as shown below.
Important messages by an important person
-- Important person [#footnote]_
.. [#footnote] Quote of an important person.
.. |H2O| replace:: H\ :sub:`2`\ O
.. |E=mc2| replace:: E=mc\ :sup:`2`Code blocks and inline code
You can provide inline and block code examples as well. reStructuredText comes with some additional functionalities that allow you to add syntax highlighting or adding line numbering. There is a concept of environments that will be used over and over again, which starts with two dots (..) followed by the environment name (in this case, code). Additional arguments to this environment are provided on the next line(s). This is shown below for the code environment.
Note, that some of these additional arguments may not always be used consistently across different reStructuredText processors. I have found that sometimes :linenos: works to generate the line numbers, and sometimes :number-lines:. It will, unfortunately, depend on your text processing engine. Keep this in mind in case you stumble across this in the future.
To display code on the same line, you can use ``dashes`` to highlight code differently from the written text. If you want to show entire code sections, start a new code environment with the ``.. code::`` block (make sure the leave an empty line between this statement and your actual code). You can optionally provide syntax highlighting hints, as well as additional options, such as using line numbers.
Example of code block without syntax highlighting:
.. code::
#include <iostream>
int main() {
std::cout << "no syntax highlighting applied" << std::endl;
return 0;
}
Same code example, but now we use C++ syntax highlighting and line line numbers:
.. code:: cpp
:linenos:
#include <iostream>
int main() {
std::cout << "c++ syntax highlighting applied" << std::endl;
return 0;
}Equations
Equations follow a similar pattern to code; we can either display them inline using the :math:`equation-goes-here` syntax, where any valid LaTeX expression may be used to write out the equation or use the .. math:: environment. An example is given below:
Equations in reStructuredText are rendered by placing a valid LaTeX math expression within a ``:math:`` block. For example, the divergence-free constraint of an incompressible flow can be stated as :math:`\nabla\cdot\mathrm{\mathbf{u}}=0`. We can also display longer equations in their own block using a dedicated `math` block on a new line (similar to how we used a code block) as:
.. math::
\frac{\partial \mathrm{\mathbf{u}}}{\partial t} + (\mathrm{\mathbf{u}}\cdot\nabla)\mathrm{\mathbf{u}} = -\frac{1}{\rho}\nabla p + \nu \nabla^2\mathrm{\mathbf{u}}Lists
Just like with Markdown, we can create itemised or enumerated lists with reStructuredText. Unlike Markdown, though, reStructuredText does provide the functionality to automatically number your enumerated lists, which can be handy if you are dealing with dynamic lists (such as todo lists, for example), where you frequently add/remove items. This is exemplified below:
We have two options, either we can add an enumerated or an itemised list. Examples of both are given below:
An itemised list may be given as:
* First list item
* Second list item
- We can have nested items as well, just make sure to leave an empty line between the child and parent item
An enumerated list may be given as:
1. First list item
2. Second list item
#. Can't be bothered writing out the number, let reStructuredText figure out what number should go at the front
#. More auto numbering
#. Nested item, use at least 3 leading spacesTables
Tables are handled rather nicely in reStructuredText. It does provide you with two options, either write grid tables or simple tables. grid tables are visually the easiest to grasp, as you can see in ASCII form what your rendered table will look like. It does have some more functionality than simple tables, though unless you are going for some fancy table formatting, both simple tables and grid tables will likely get the job done (with the advantage of simple tables having an easier-to-write syntax).
We can use two different styles with reStructuredText. Either, we can use the more complete syntax of ``grid tables`` or the easier to write, but limited in functionality, ``simple tables``. An example for both is shown below:
Grid table:
+------------+------------+-----------+
| Header 1 | Header 2 | Header 3 |
+============+============+===========+
| body row 1 | column 2 | column 3 |
+------------+------------+-----------+
| body row 2 | Cells may span columns.|
+------------+------------+-----------+
| body row 3 | Cells may | - Cells |
+------------+ span rows. | - contain |
| body row 4 | | - blocks. |
+------------+------------+-----------+
Simple table:
===== ===== ======
Inputs Output
------------ ------
A B A or B
===== ===== ======
False False False
True False True
False True True
True True True
===== ===== ======Links and images
Links are simply put between two dashes, which start with a description of the link (i.e. the link text) followed by the link itself within angle brackets and an underscore as in `link text, can have spaces <https://www.important-link.com>`_.
Images, on the other hand, are handled differently and are placed within their own environment, just like code and math. This is different from Markdown, where there is a close resemblance between links and images. Placing images in their own environment allows them to provide some additional arguments, such as the width of the image and the alternative (alt) text.
Links can be added using a description for a website or URL followed by the website itself. Both of these have to be given inside dashes and followed by an underline
Example: `Link to cfd.unversity <https://cfd.university>`_
Images are provided using the image environment and are similar to how we added equations and code. We can, again, provide additional information such as the width of the image and the alternative description (used for screen readers). An example is given below:
.. image:: http://cfd.university/wp-content/uploads/resources/documentation/logo.png
:width: 800
:alt: cfd university logoAdding badges
And, finally, we looked at adding badges from shields.io to our document. I won’t go into detail here about what shields.io is doing, but you can read up on my previous article which contains a gentle introduction to badges. These will become important again in future series.
.. image:: https://img.shields.io/badge/Version-1.2.5-red
:alt: Version number
.. image:: https://img.shields.io/badge/License-MIT-blue
:alt: MIT licenseUsing Sphinx to manage your user guide
When it comes to writing the user guide, we have a few additional requirements over the 1-page readme file. The user guide will typically contain several pages, all of which may be longer than the single 1-page readme file. So we want to have, at a bare minimum, a way to automatically generate the table of contents and some form of navigation through all these pages. If you have a rather long user guide, you may also want to be able to search it.
reStructuredText is, just like Markdown, a markup language and does not provide support for any of the above. But some packages support you doing all of the above (and more) and one of these packages, arguably the most used one, is Sphinx. I like Sphinx because I can never remember how to set up a project and generate documentation, but it is so intuitive that I don’t have to. Sphinx comes with an automated script that sets up everything for you, you just have to provide details about the project as some inputs.
Creating your documentation once everything is set up correctly is also pretty straightforward and can be integrated with build scripts such as CMake.
Let’s look at the installation process first, how to set up a starter project, and finally how to change the look and feel with custom themes. We’ll use that starter project and turn our 1-page readme file of the mesh reading library into a user guide.
Installing Sphinx
Sphinx is a pure Python package (we have been using the term library thus far in the context of C++, packages are nothing else than libraries but Python prefers the term package). This means we have to ensure Python is installed. This is, thankfully, pretty straightforward and so I trust that you will be able to achieve this. All you need is the Python download website, download the latest release for your operating system and you are ready.
Python comes with a package manager called pip, that allows you to download packages/libraries on the fly. It is very similar to Conan, the C++ package manager we looked at a while back. To ensure that pip is installed, run the following command:
On Windows:
py -m pip --versionOn macOS/Unix:
python3 -m pip --versionThis command should return a version number for pip. If it does, you can continue, otherwise, you’ll have to install it first. You can try to do this directly via the command line using the following commands:
On Windows:
py -m ensurepip --default-pipOn macOS/Unix:
python3 -m ensurepip --default-pipIf that still does not work, consult the Python help section on installing packages with pip, which covers additional steps.
With Python and pip installed, we can go ahead and install Sphinx using the following command:
On Windows:
py -m pip install sphinxOn macOS/Unix:
python3 -m pip install sphinxIf you don’t have administrator rights on your PC, you can install Python packages to your user’s home directory instead. To do that, supply pip with the --user flag, i.e. pip install --user sphinx. To test that Sphinx was indeed installed correctly, you should run the following command after the installation and you should see the version of Sphinx that you have just installed:
sphinx-build --versionCreating a starter project
To get started with Sphinx, or indeed any other tool of code documentation, let’s first review the default structure of a library or software project that is trialled and tested and that we should adopt in our projects as well.
root
├── docs/
├── src/
│ └── ...
├── tests/
│ └── ...
├── buildScript.ps1
└── buildScript.shLooking at this structure, we have already seen the src/ directory extensively, i.e. this is where we keep our source code and its header files. We looked at the tests/ directory in my series on software testing and saw that this contains our unit, integration, and system (or end-to-end) tests. Ignoring the build scripts for now, we have gained a new folder called docs/ and this is where all of our documentation will be written and processed into a useful output format.
Using either your developer PowerShell on Windows, or your terminal on macOS/Unix, navigate to the docs/ folder and type the following:
sphinx-quickstartThis will walk you through the automatic tool to set up your user guide environment. I have found that this script can change over time a bit so I don’t go through each question it asks you, but these are typically the sort of inputs you would provide:
- Whether you want to separate your build and source files. Unless you have strong reasons to do otherwise, I would always recommend separating them.
- The name of the software project you want to provide documentation for.
- The author’s name of the project.
- Project releases. This can be left empty, or you could specify a specific version if you want to such as 1.3.1.
- The language used for the documentation. Typically English (
en), but you may want to write in your native language, then you would change this.
After you have gone through the quickstart, you should have generated the following structure within your docs/ folder:
docs/
├── build/
├── source/
│ ├── _static/
│ ├── _templates/
│ ├── conf.py
│ └── index.rst
├── make.bat
└── MakefileThe build/ folder will contain our output once we run Sphinx on our input files. The source/ folder contains all of our inputs in the form of reStructuredText files. We can see that the index.rst file has already been generated for us. If we open this, we’ll see the following output:
.. meshReaderLib documentation master file, created by
sphinx-quickstart on Thu Apr 11 05:12:53 2024.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to meshReaderLib's documentation!
=========================================
.. toctree::
:maxdepth: 2
:caption: Contents:
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
You’ll see that I have gone ahead and started the process of writing the mesh reading lib documentation, i.e. I have named the project meshReaderLib. This is the file that will be used to generate the entry into our user guide, and you can see the .. toctree:: environment, with a maximum depth of 2 that will generate the table of contents for us automatically. As the help text at the start suggests, we can completely redesign this file, but it would make sense to retain the .. toctree:: environment at a minimum.
Leaving a new line after the :caption:, we can start writing the filenames of reStructuredText files we want to include in the table of contents. Say we have a file called introduction.rst and this file is located in the same directory as the index.rst file, then we would simply add the word introduction (no file ending) on line 13, i.e.
.. toctree::
:maxdepth: 2
:caption: Contents:
introductionWe can do this with as many files as we want, and thus break our user guide documentation into separate files. This will help us manage longer documents while still including all of them in the table of contents and the user guide as a whole (i.e. all separate files will be considered when searching the project or creating additional elements such as the index).
Let’s do that and create a file called introduction.rst and just fill that with some example text. I have created that file and essentially copied all of the reStructuredText examples from above into it, separated by headings.
To generate the documentation from this, you will then have to run, from within the docs/ folder, i.e. you need to be within this folder which contains the main.bat and Makefile files, the following command:
make htmlThis will generate your user guide in HTML format from all of your reStructuredText files, i.e. here index.rst and introduction.rst. Other targets are available, but HTML is the one we will most often use.
You will notice that your build/ directory now contains some output, i.e. it has changed to
docs/
├── build/
│ ├── doctrees/
│ │ └── ...
│ └── html/
│ ├── index.html
│ └── ...
├── source/
│ ├── _static/
│ ├── _templates/
│ ├── conf.py
│ ├── index.rst
│ └── introduction.rst
├── make.bat
└── MakefileWe can ignore the doctrees/ folder for now, but notice that within the build/ folder we have the html/ folder, which contains a few HTML files, among which we are interested in the index.html file. This is our entry point and if you open this, you’ll be able to navigate through the content we have provided in the reStructuredText files but now rendered as HTML.
If you want to to inspect the created documentation, I have uploaded it here to the website for you to inspect, you can use the links below to see the individual files:
- index.rst: Look at the index file where we have just changed the additional file to include in the table of contents.
- introduction.rst: Look at the file which contains all of the examples above that we went through when introducing reStructuredText.
- index.html: See how the previous two files are finally rendered into HTML. Have a look around to see what Sphinx provide you out of the box, e.g. searching functionality.
Changing the look and feel with custom themes
The default theme that comes with Sphinx is already decent, but since it is just a collection of HTML and CSS, it is easy to extend and create custom themes and so people have done just that. To start with, I would not recommend embarking on a journey trying to adjust the CSS file to your liking unless you have extensive front-end web development experiences under your belt, but instead use themes provided by others.
A good resource to find some nice themes is provided over at Write the docs. You’ll find a selection of well-maintained and good-looking templates, where the most popular one is the one provided by Read the docs (which is the first theme provided). If you click through to its GitHub page, where the theme is maintained, you’ll see that to install this theme, we can use pip for that as well:
On Windows:
py -m pip install sphinx-rtd-themeOn macOS/Unix:
python3 -m pip install sphinx-rtd-themeThen, to use this theme, we have to change the theme from alabaster to sphinx_rtd_theme within the docs/source/conf.py file. Towards the bottom of the file, you find the html_theme variable and we set this now to:
html_theme = "sphinx_rtd_theme"Create the documentation again by running the make, i.e.
make htmlIf you open the docs/build/html/index.html now in your browser, you should see the read the docs theme applied to your documentation. To see a preview, I have uploaded the same HTML-generated documentation again with the updated theme so you can have an interactive look at it.
Creating a user guide for the meshReaderLibrary
At this point, we have all the things in place to create the user guide from our lengthy 1-page readme file that we created in the previous article. Since it is written in Markdown, we first have to translate it to reStructuredText. We could, of course, use our newfound knowledge of this markup language and do this ourselves, but there are tools available that can do that for us, and our time is probably more valuable than doing something that can be done in seconds automatically.
I have used alldocs.app, which lets you upload a Markdown file and then convert it to reStructuredText. It is quite decent and does the job well, but feel free to look around for alternatives. Either way, once we have created the reStructuredText file from our Markdown template, we can simulate writing a user guide by splitting this 1-page file into separate files.
We have three main sections in our previous 1-page readme file; introduction, installation, and usage. I have created these three files and copied the corresponding content from the 1-page readme file into them. Then, we have to modify our table of contents within the index.rst file as follows:
.. toctree::
:maxdepth: 5
:caption: Contents:
introduction
installation
usageApart from adding the files so that Sphinx can create the documentation correctly, I’ve also increased the max-depth argument, which lets us display headings in the table of contents up to 5 levels. We actually have only 3 levels of section headings, but as our user guide grows, we may have more levels that we want to display. If it gets too cluttered, just decrease the max-depth argument until the table of contents looks good.
After running the make file, i.e. make html from within the docs/ folder, you should have now the following structure (not all files are shown but only the ones that are of importance):
docs/
├── build/
│ ├── doctrees/
│ │ └── ...
│ └── html/
│ ├── index.html
│ └── ...
├── source/
│ ├── _static/
│ ├── _templates/
│ ├── conf.py
│ ├── index.rst
│ ├── introduction.rst
│ ├── installation.rst
│ └── usage.rst
├── make.bat
└── MakefileAnd, if everything worked out correctly, you should have created the user guide for the mesh reading library, just like we did before with the other dummy user guide above.
I have uploaded that user guide here for you to inspect online. If you want to inspect any of the *.rst files, locate the View page source link which you can find at the top of the document. Try out the search functionality and how you can navigate through the document, you’ll see that Sphinx has done a decent job of putting a nice structure around the entire documentation. Oh, and as you probably have noticed, I used the Read the docs theme again.
Summary
Writing documentation is not the most interesting thing in the world, and it is usually one of a programmer’s more dreaded tasks. But I hope to have shown you in this article that writing a decent user guide does not take that much time and we can rely on tools that make our life easier.
Using Sphinx, we have seen that creating a full-blown user guide completed with search, navigation, and table of contents functionality is a piece of cake. While we have only looked at its capabilities to create an HTML output, there is much more to Sphinx and it is able to create various output formats for us.
Sphinx relies on a markup language that we know as reStructuredText, which presents a much more consistent markup language than Markdown, which we investigated in our previous article on writing a 1-page readme file. reStructuredText and Markdown, especially its extended specification provided by Markdown, come with similar functionality, but reStructuredText does come with its own processing engine (Docutils) which ensures a consistent markup syntax (which Markdown lacks).
As a general rule of thumb, start writing your 1-page readme file and then switch to a user guide as your project grows. There is a saying in the software development community that the text you write within your documentation is just as important as the text you write in your source files, i.e. the code itself. Getting this aspect right and providing high-quality documentation will ensure uptake of your software and happy users overall.