In this article, we have a bit of ground to cover. We will look at how we can take the library code that we developed over the last few articles and turn it into either a static, dynamic, or header-only library. While static and dynamic libraries only differ in the compilation process, we have to modify our code completely for header-only libraries and we will see how to do that. Dynamic libraries require some further fine-tuning on Windows and we will highlight how to get dynamic libraries to work on Windows, something not very well covered elsewhere. We’ll test our library as well and see that we get converged results for our 1D heat diffusion equation in the end.
Download Resources
All developed code and resources in this article are available for download. If you are encountering issues running any of the scripts, please refer to the instructions for running scripts downloaded from this website.
In this series
- Part 1: Understanding static, dynamic, and header-only C++ libraries
- Part 2: How to write a CFD library: Discretising the model equation
- Part 3: How to write a CFD library: Basic library structure
- Part 4: How to write a CFD library: The Conjugate Gradient class
- Part 5: How to write a CFD library: The vector class
- Part 6: How to write a CFD library: The sparse matrix class
- Part 7: How to write a CFD library: Compiling and testing
- Part 8: How to integrate external CFD libraries in your code
- Part 9: How to handle C++ libraries with a package manager
In this article
How we got here
Let’s recap how we ended up at this point, as there is quite a bit to remember. We have gone through a few articles all looking at different aspects, and I’ll briefly summarise each article below so we have a shared understanding of what we have done. This will make it easier to follow the rest of the article.
- How to write a CFD library: Discretising the model equation: In this article, we looked at our model problem. We set out to solve the 1D heat diffusion equation, and we discretised the equation. In doing so, we realised that we would need an iterative matrix solver to solve the linear system of equations of the form \mathbf{Ax}=\mathbf{b}.
- How to write a CFD library: Basic library structure: In this article, we looked at how we would have to structure our library and how to compile it for testing. We wrote a test that implemented the 1D heat diffusion equation that we will use here as a test. We will also revisit the compilation step in this article and make it more general.
- How to write a CFD library: The Conjugate Gradient class: Here, we looked at the base class for the linear algebra solver and then derived a single child class that implemented the Conjugate Gradient algorithm. We highlighted the need for matrix-vector-scalar multiplications, providing the necessary motivation to provide our own matrix and vector classes.
- How to write a CFD library: The vector class: This article, then, implemented the vector class and focused primarily on mathematical arithmetic operations, and how we can implement these for the vector class using operator overloading.
- How to write a CFD library: The sparse matrix class: The final piece in the puzzle was the matrix class. We spend quite a bit of time understanding how we can implement sparse matrices, which dominate CFD applications. We looked at the implementation of a sparse matrix class itself using the compressed sparse row (CSR) storage format.
With all of this out of the way, we are now finally in a position to look at libraries themselves. What I want to do in this article specifically is to show you how you can turn this library code now into a static, dynamic, or header-only library. We discussed these different types earlier in this series in the article Understanding static, dynamic, and header-only C++ libraries, which you may want to check out if you need a refresher. If you want the TL;DR version, the image below summarises how these different libraries are injected into our code.
Header-only libraries do not define any source code files and thus do not get compiled into a library itself. All function definitions live in header files, which can simply be included in a project. It’s the simplest form of a library and is especially useful for pure C++ libraries that make a lot of use of templates.
Static libraries do have source files (i.e. files ending in *.cpp, typically), and these get compiled into object codes first, and then packaged into a library. The best analogy for static libraries is a zip file; all your compiled code is put together into a single library (or zip file) and that gets injected into your executable at compile time if you want to bring this library into your codebase.
Dynamic libraries are compiled similarly to static libraries, but they don’t get compiled into any executable. The idea is that your executable has to look for the dynamic library at runtime, meaning that even if you successfully compile your application against a dynamic library, there is still no guarantee that you will be able to run your code after compilation. There may be look-up issues at runtime and dynamic libraries are the most difficult ones to get to work.
All of these libraries have their advantages and disadvantages, which we looked at in the article linked above, but at this point, we want to move on and see how we can compile the libraries into their different forms and use them in our 1D heat diffusion example code. We’ll start with static and dynamic libraries, as their compilation step is rather straightforward. For header-only libraries, we have to make changes to the code first, which we will look at separately.
This website exists to create a community of like-minded CFD enthusiasts and I’d love to start a discussion with you. If you would like to be part of it, sign up using the link below and you will receive my OpenFOAM quick reference guide, as well as my guide on Tools every CFD developer needs for free.
Join now
Compiling the library as a static or dynamic library
Let’s split this part into two subsections. First, we’ll explore how we can compile our library on a UNIX machine (e.g. Ubuntu, macOS, etc.), and then we look at Windows, which has some nasty surprises for us in store. UNIX platforms are usually quite good for developing code and are preferred by most in the CFD community. After reading this section you may understand why. I want to break free of this UNIX mindset, though, and show that Windows can be used as well (depending on how much pain you can tolerate).
Before we get started, we are going to add a few files to our project. The new files with the new project structure is shown below.
root
├── build
├── linearAlgebraLib
│ ├── linearAlgebraLib.hpp
│ └── src
│ ├── vector.(hpp|cpp)
│ ├── sparseMatrixCSR.(hpp|cpp)
│ ├── linearAlgebraSolverBase.(hpp|cpp)
│ └── conjugateGradient.(hpp|cpp)
├── main.cpp
├── buildAndRun.(sh|bat)
├── testStaticLib.(sh|bat)
└── testDynamicLib.(sh|bat)We have added the testStaticLib.(sh|bat) and testDynamicLib.(sh|bat) files in the root directory of our project, where *.sh files will be used on UNIX (Ubuntu, other Linux distributions, macOS) and *.bat files on Windows. These will be used to build and test our static and dynamic libraries, respectively. Let’s do exactly that next.
Compiling on UNIX platforms
Before we look at the static and dynamic library build scripts, let’s look back at the original buildAndRun.sh file. For convenience, this is listed below again:
#!/bin/bash
rm -rf build
mkdir -p build
# compile
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/sparseMatrixCSR.cpp -o build/sparseMatrixCSR.o
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/vector.cpp -o build/vector.o
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/linearAlgebraSolverBase.cpp -o build/linearAlgebraSolver.o
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/conjugateGradient.cpp -o build/conjugateGradient.o
g++ -c -g -O0 -Wall -Wextra -I. main.cpp -o build/main.o
# link
g++ -o build/main.out build/sparseMatrixCSR.o build/vector.o build/linearAlgebraSolver.o build/conjugateGradient.o build/main.o
# run
./build/main.outAs we have seen before, we compile our code with the GNU g++ compiler, and we do so for each source file (*.cpp) as seen in lines 7-11. However, what I want to focus on is line 14. On this line, we build an executable that directly includes all object files (.o) from our library code. This is not what we want, though. The whole point of a library is that we want to inject that into our code, instead of having to compile it as part of our project and then inject the object files from the library into our code. So we will have to separate line 14 into the creation of a library and then link that library with our program, here the main.cpp file.
Static library build script
Let’s look at the new build script and then discuss the changes made to it. Below is the content for the testStaticLib.sh file.
#!/bin/bash
rm -rf build
mkdir -p build
# compile source files into object code
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/sparseMatrixCSR.cpp -o build/sparseMatrixCSR.o
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/vector.cpp -o build/vector.o
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/linearAlgebraSolverBase.cpp -o build/linearAlgebraSolver.o
g++ -c -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/conjugateGradient.cpp -o build/conjugateGradient.o
g++ -c -g -O0 -Wall -Wextra -I. main.cpp -o build/main.o
# link object files into static library
ar rcs build/libStaticLinearAlgebra.a build/sparseMatrixCSR.o build/vector.o build/linearAlgebraSolver.o build/conjugateGradient.o
# compile main function and link library into executable
g++ -g -O0 -Wall -Wextra -I. build/main.o -o build/staticLibExample -Lbuild -lStaticLinearAlgebra
# remove object files
rm build/*.o
# run
echo "Running static library example"
./build/staticLibExampleThe compilation on lines 7-11 is unaffected, i.e. we still produce object files in the same way as we did before. Line 14, though, has changed, where we now call a program called ar, which is the archiver on UNIX systems. The command line flags rcs are used to essentially allow overwriting or updating an existing library if already present and new files were generated (source code changed).
The first argument is the location and filename where we want to store our static library, in this case, in the build directory. Notice that on UNIX system we use a file ending *.a to indicate that it is a static library (or archive). The following arguments are all the object files that should be included in this archive (or static library).
Then, on line 17, we simply build an executable called staticLibExample within the build directory, and the only object file we are using is the main.o file. However, we now also specify the location of the static library with the -L flag, and then the name of the static library with the -l flag (notice the absence of spaces between the flags and their arguments). On UNIX platforms, the name of the library is expected to start with lib and end in the ending *.a, both of which are implicitly added when you specify the library name to the -l flag. Thus, the names we specify here and on line 14 are different.
If we think of the analogy that the archiver ar just creates glorified zip files, then we can see that specifying the location of the static library along with its name allows extracting the archive, which just contains the object files of the library code we packaged on line 14. We essentially recover again what was line 14 in the buildAndRun.sh script we saw before, where we explicitly linked against all object files. Now, we sort of implicitly link against them.
The final lines 20 and 24 remove all object files and run the executable to test that the compilation has worked.
Dynamic library build script
The dynamic library creation follows a similar step, i.e. we separate the creation of the library into a separate step and then link against it in our main program. Let’s look at our build script and then analyse it again step-by-step.
#!/bin/bash
rm -rf build
mkdir -p build
# compile source files into object code
g++ -c -fPIC -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/sparseMatrixCSR.cpp -o build/sparseMatrixCSR.o
g++ -c -fPIC -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/vector.cpp -o build/vector.o
g++ -c -fPIC -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/linearAlgebraSolverBase.cpp -o build/linearAlgebraSolver.o
g++ -c -fPIC -g -O0 -Wall -Wextra -I. linearAlgebraLib/src/conjugateGradient.cpp -o build/conjugateGradient.o
g++ -c -g -O0 -Wall -Wextra -I. main.cpp -o build/main.o
# link object files into dynamic library
g++ -shared -g -O0 -Wall -Wextra -I. -o build/libDynamicLinearAlgebra.so build/sparseMatrixCSR.o build/vector.o build/linearAlgebraSolver.o build/conjugateGradient.o
# compile main function and link library into executable
g++ -g -O0 -Wall -Wextra -I. build/main.o -o build/dynamicLibExample -Lbuild -Wl,-rpath=build -lDynamicLinearAlgebra
# remove object files
rm build/*.o
# run
echo "Running dynamic library example"
./build/dynamicLibExampleLines 7-11 are almost the same compared to our previous build scrips, but we have added the -fPIC compiler flag. PIC stands for position independent code, and we looked at this briefly in our overview on static, dynamic, and header-only libraries. This is a requirement for dynamic libraries, as we don’t know in advance where in memory these functions will be (as opposed to static libraries, where we do know that at compile time as we inject all object files directly into our code). We only add it to our library code, but not the main.cpp file.
Then, on line 14, we use now g++, i.e. the compiler, to link all object files (*.o) from our library code into a dynamic library. Here the compiler flag -shared tells g++ to create the dynamic library, and we put that into the build directory with a file ending of *.so, standing for shared object. I say we use g++, that is incorrect, we abuse it here and g++ will call for us the linker, which on UNIX platforms is called ld. But this is common practice, as g++ will pass a lot of boilerplate instructions to ld which we don’t have to type. If you need more convincing that this is a good idea, check out this book chapter, pages 4-5.
The linking of the main program on line 17 is now somewhat more interesting. We still pass the -L and -l flag to g++ (and this will be forwarded again to the linker ld) and they operate in the same way as for static libraries. However, since dynamic libraries are only loaded during runtime, not during compilation, we also pass the ominous -Wl,-rpath=build flag. This flag does two things, first, the -Wl part says that the following flags should be passed to the linker ld directly. Any flag listed after the comma will be used by the linker.
The -rpath=build adds the build directory to the run path. Our executable will then have some idea of where to look for the dynamic library during runtime. If you don’t specify this, you can still compile your code, but when you try to run it, you’ll get an error message when you try to execute your code saying: error while loading shared libraries: libDynamicLinearAlgebra.so: cannot open shared object file: No such file or directory. If you encounter this error, it means your program does not know where it is located and by adding the -rpath=build variable, your code will know where to check and find the library now.
Compiling on Windows platforms
Let’s have a quick look at the buildAndTest.bat program as well before we look into creating the static and dynamic library. We saw previously that it is given as:
@echo off
del /q build
mkdir build
REM compile
cl /nologo /c /EHsc /Zi /Od /I. /Fobuild\sparseMatrixCSR.obj linearAlgebraLib\src\sparseMatrixCSR.cpp
cl /nologo /c /EHsc /Zi /Od /I. /Fobuild\vector.obj linearAlgebraLib\src\vector.cpp
cl /nologo /c /EHsc /Zi /Od /I. /Fobuild\linearAlgebraSolver.obj linearAlgebraLib\src\linearAlgebraSolverBase.cpp
cl /nologo /c /EHsc /Zi /Od /I. /Fobuild\conjugateGradient.obj linearAlgebraLib\src\conjugateGradient.cpp
cl /nologo /c /EHsc /Zi /Od /I. /Fobuild\main.obj main.cpp
REM link
cl /Fe:build\main.exe build\sparseMatrixCSR.obj build\vector.obj build\linearAlgebraSolver.obj build\conjugateGradient.obj build\main.obj
REM run
build\main.exeIn the following, we’ll attempt to turn this into a static and dynamic library as well, as we just did for the UNIX build script.
Static library build script
Below is the version that will allow us to build and execute our static library on Windows. Remember, in order to be able to compile code on Windows, you have to use a PowerShell that has the Visual Studio variables loaded. I went over this in our basic library structure article when discussing the above build script for Windows, scan that section if you need a refresher.
@echo off
del /q build
mkdir build
REM compile source files into object code
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILELIB linearAlgebraLib\src\sparseMatrixCSR.cpp /Fobuild\sparseMatrixCSR.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILELIB linearAlgebraLib\src\vector.cpp /Fobuild\vector.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILELIB linearAlgebraLib\src\linearAlgebraSolverBase.cpp /Fobuild\linearAlgebraSolver.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILELIB linearAlgebraLib\src\conjugateGradient.cpp /Fobuild\conjugateGradient.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILELIB main.cpp /Fobuild\main.obj
REM link object files into static library
lib /nologo /OUT:build\staticLinearAlgebra.lib build\sparseMatrixCSR.obj build\vector.obj build\linearAlgebraSolver.obj build\conjugateGradient.obj
REM link library into main executable
cl /nologo /EHsc /Od /I. build\main.obj /Fe:build\staticLibExample.exe /link /LIBPATH:build staticLinearAlgebra.lib
REM remove object files
del build\*.obj
REM run
echo Running static library example
build\staticLibExample.exeWe compile our code on lines 7-11 as before, and even line 14 does not look too different from line 14 in the UNIX build script for static libraries. We have included, though, a preprocessor directive with the /D flag, here /DCOMPILELIB, and we will see in the next section why we have to do this. We do this not just for the library code, but also for the main.cpp file.
We use the lib command here, instead of ar which is only available on UNIX. We specify the location and name of the library, followed by all object files that should be stored in this static library (archive). We then use it in a similar way on line 17, where we have added the /link flag to indicate that the following arguments are linker commands. On both lines, notice that our library no longer starts with the prefix lib as it did on our UNIX build script(i.e. our library is now called staticLinearAlgebra and not libStaticLinearAlgebra). This is not required on Windows and is also not a convention, so we removed it here to follow standard practices.
The /LIBPATH: behaves the same way as the -L flag on UNIX, i.e. it states where the library should be built into. It is followed by the name of the static library, and you see that on Windows the names of the library given on lines 14 and 17 are the same, i.e. we are not implicitly adding any strings to the library name as we did on UNIX.
Lines 20 and 24, again, remove object files we no longer need and execute our code for testing.
Dynamic library build script
I once read a great quote, which states (I am paraphrasing here as I don’t remember the exact wording):
Compiling on Windows is only difficult if you expect it to be a direct clone of UNIX
anonymous
So, do you really want to learn how to turn your library into a dynamic library to be run on Windows? Yes? Well, then strap in, we have some work to do. Getting things done on Windows is not as straightforward as we would hope. However, I can pretty much guarantee you that you won’t find this information easily anywhere else; if you look to build dynamic libraries on Windows, you’ll find plenty of resources on how to do that with CMake or Visual Studio. But if you wanted to know how to do that on the command line yourself, there is pretty much nothing on that topic.
I’d argue you should know how to do things first yourself, on the command line, before upgrading to a build system. In this way, you can troubleshoot errors easily when they come up and are not dependent on ChatGPT to tell you what’s wrong (in most cases, ChatGPT is rather unhelpful in these situations). So, let’s look then how to build a dynamic library ourselves, without tool support, we’ll later learn how to achieve this with CMake as well. Let’s start with the build script and see what we are doing here:
@echo off
del /q build
mkdir build
REM compile source files into object code
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILEDLL linearAlgebraLib\src\sparseMatrixCSR.cpp /Fobuild\sparseMatrixCSR.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILEDLL linearAlgebraLib\src\vector.cpp /Fobuild\vector.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILEDLL linearAlgebraLib\src\linearAlgebraSolverBase.cpp /Fobuild\linearAlgebraSolver.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILEDLL linearAlgebraLib\src\conjugateGradient.cpp /Fobuild\conjugateGradient.obj
cl /nologo /Zi /c /EHsc /Od /I. /DCOMPILEDLL main.cpp /Fobuild\main.obj
REM link object files into dynamic library
link /nologo /DLL /OUT:build\dynamicLinearAlgebra.dll build\sparseMatrixCSR.obj build\vector.obj build\linearAlgebraSolver.obj build\conjugateGradient.obj
REM link library into executable
cl /nologo /EHsc /Od /I. build\main.obj /Fe:build\dynamicLibExample.exe /link /LIBPATH:build build\dynamicLinearAlgebra.lib
REM remove object files
del build\*.obj
REM run
echo Running dynamic library example
build\dynamicLibExample.exeLines 7-11 again compile the code, but we do not have to specify that this is position-independent code here. We also make use of the preprocessor directive /DCOMPILEDLL, i.e. we are defining the COMPILEDLL here and this will help us distinguish later if we want to compile a static or dynamic library.
On line 14, we call the linker directly and say that we want to create a dynamic link library (DLL), which is Windows terminology for dynamic library, or shared object, as known on UNIX systems. This follows the usual pattern, i.e. we specify first the name and location of the dynamic library, which is followed by the source files it depends on.
We then link the generated library on line 17 against our compiled main function and then remove build files on line 20, as well as execute the code on line 24 for testing, as per usual. Did you notice something strange? On line 14, we create the library dynamicLinearAlgebra.dll but then we link against dynamicLinearAlgebra.lib on line 17, i.e. the postfix has changed from *.dll to *.lib. Windows creates two libraries for us; a *.lib and a *.dll file. The *.dll file contains the actual library while the *.lib file acts like a wrapper around the dynamic library. We call (and link against) the *.lib file which will retrieve the correct functions from the *.dll file.
Intermission: The preprocessor
Remember that for both the static and dynamic libraries, we introduced a preprocessor directive with the /D compiler flag. Now it is time to consume this compiler flag. Before we do that, though, let’s look at a simple example to understand what preprocessor directives are doing. For a more in-depth discussion, I can only recommend learncpp again on this topic, they have a great write-up, as always.
Let’s look at the following code and assume this file is called main.cpp:
// main.cpp
#include <iostream>
int main() {
#ifdef PRINT_LOGS
std::cout << "printing logging information ... " << std::endl;
#endif
#ifdef WIN32
std::cout << "I am compiled on windows" << std::endl;
#else
std::cout << "I am not compiled on windows" << std::endl;
#endif
#ifdef __cplusplus
std::cout << "i am compiled with a c++ compiler" << std::endl;
#endif
return 0;
}Preprocessor directives start with the # character. We saw that before in our classes when we declared #pragma once. As a reminder, #pragma once makes sure that each header file is only included once during compilation, even if it is included in several files with an #include statement. If it was included several times, we get a compiler error saying that we have multiple definitions of the same class, because we included that file multiple times during compilation.
Whenever you see code that starts with a preprocessor directive (# character), then what will happen is that before your compiler sees the source code for compilation, the preprocessor will go over the file and apply all of these statements. What does that mean? Let’s look at lines 5-7. We are saying that if PRINT_LOGS is defined (#ifdef is a shorthand notation for #if defined(PRINT_LOGS)), then go ahead and include everything up until the next #endif statement. If it isn’t defined, then remove the entire code. How do we define PRINT_LOGS? During compilation with the /D flag (or -D on g++).
So, if we compiled the above code, assumed to be called main.cpp, with cl /DPRINT_LOGS main.cpp on Windows or g++ -DPRINT_LOGS main.cpp on UNIX, our executable would print printing logging information … to the console. Without these definitions, it would simply remove this line.
On lines 9-13 and 15-17 we make use of some system definition. For example, on line 9, we check if we are compiling on Windows, and if we are, we print the statement on line 10. If it is not Windows, then we print line 12. Similarly, we can figure out if we used a C++ compiler on line 15, which can be handy if we want to compile both with a c and C++ compiler.
Necessary library modifications
Ok, so we do understand the preprocessor now. We need that knowledge in a second, but first, we need to understand what we want to achieve. On UNIX, all functions are exported in a way that a dynamic library can consume them. On Windows, unfortunately, it is different for compiling and linking.
Say we are looking at one function of our matrix class void set(unsigned row, unsigned column, double value);. If we compile the code now as a dynamic library, Windows expects us to say which function should be part of the dynamic link library (*.dll file). We do that by exporting that function to the library, and this will then generate the required code in our *.lib library, which we use to make calls to the *.dll library. If we do not export any functions from the *.dll file, then Windows will not generate the *.lib function (as there is nothing to call) and we can’t proceed with the linking.
We specify a function to be exported through a Windows-specific command __declspec(dllexport). So our function would now be __declspec(dllexport) void set(unsigned row, unsigned column, double value);. However, during linking, we don’t want to export this function anymore but rather import this function from the generated *.dll file, for which we use __declspec(dllimport). So, during linking, we are expecting a statement of __declspec(dllimport) void set(unsigned row, unsigned column, double value);. This allows us on Windows to specify which functions to export, while on UNIX all functions are exported by default.
So, hopefully, you can see the link now between the preprocessor and the above export/import conditional statement. What we have to do now is modify all of our header files and include the following code directly after #pragma once:
#if defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(__NT__)
#if defined(COMPILEDLL)
#define LINEARALGEBRALIB_EXPORT __declspec(dllexport)
#elif defined(COMPILELIB)
#define LINEARALGEBRALIB_EXPORT
#else
#define LINEARALGEBRALIB_EXPORT __declspec(dllimport)
#endif
#else
#define LINEARALGEBRALIB_EXPORT
#endifThe first line checks if we are on Windows. Some systems define it differently, so we try to capture all possible Windows identifications here. Then, line 2 checks if the COMPILEDLL is defined, which is the case when we are trying to build the dynamic library. If this is the case, we define a new variable called LINEARALGEBRALIB_EXPORT (i.e. the convention is to use your library name + _EXPORT) and set it equal to the __declspec(dllexport) we saw before. On the other hand, if we compile a static library (COMPILELIB, line 4), we still define LINEARALGEBRALIB_EXPORT but set it equal to an empty string (by not specifying anything after the name). If we specify neither, we set LINEARALGEBRALIB_EXPORT equal to __declspec(dllimport) and so our linker will be able to import our library.
Line 9 is essentially there for UNIX environments, i.e. if we are not on Windows, we still want to define LINEARALGEBRALIB_EXPORT but set it equal to an empty string again so that we have this variable always defined but only do something in case we want to compile a dynamic library on Windows.
With LINEARALGEBRALIB_EXPORT defined, we are now ready to change our classes. We saw before that we could now append this to all of our functions, but we can also directly inject it into our classes which means we only have to make modifications in one place. Taking the LinearAlgebraSolverBase class as an example, our modified class now looks like:
class LINEARALGEBRALIB_EXPORT LinearAlgebraSolverBase {
// ...
};You see, we just inject this new definition directly in front of the library name and that’s it. We have to now copy and paste the preprocessor statements into all header files (or if you want to be a good programmer, create a header file containing all the preprocessor statements and include it in each header file). For completeness, the LinearAlgebraSolverBase class now looks like the following:
#pragma once
#if defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(__NT__)
#if defined(COMPILEDLL)
#define LINEARALGEBRALIB_EXPORT __declspec(dllexport)
#elif defined(COMPILELIB)
#define LINEARALGEBRALIB_EXPORT
#else
#define LINEARALGEBRALIB_EXPORT __declspec(dllimport)
#endif
#else
#define LINEARALGEBRALIB_EXPORT
#endif
#include "linearAlgebraLib/src/sparseMatrixCSR.hpp"
#include "linearAlgebraLib/src/vector.hpp"
namespace linearAlgebraLib {
class LINEARALGEBRALIB_EXPORT LinearAlgebraSolverBase {
public:
LinearAlgebraSolverBase(unsigned numberOfCells);
virtual ~LinearAlgebraSolverBase() = default;
void setCoefficientMatrix(const SparseMatrixCSR &matrix);
void setRightHandSide(const Vector &rhs);
virtual Vector solve(unsigned maxIterations, double convergenceThreshold) = 0;
protected:
SparseMatrixCSR _coefficientMatrix;
Vector _rightHandSide;
};
} // namespace linearAlgebraLibWe just doubled the size of the interface, but that is what we have to do in order to get dynamic libraries to work on Windows. Now it is up to you to decide if it is worth the pain.
Finally, before we finish this section, leave it behind us and perhaps never dare speak or even think about it again, there is a good article that shows how to achieve cross-platform (Windows, UNIX) compilation of dynamic libraries using CMake. Ultimately, we want to be using a build system such as CMake as well, but for now, we have an idea of what is involved in writing cross-platform libraries ourselves.
As I mentioned earlier, you probably won’t find that knowledge condensed in this form elsewhere (which probably means no one cares and no one except me is reading this in the future), but if you are serious about CFD development, cross-platform compilation should be the norm these days. Don’t limit yourself to running your code on UNIX only, be bold, and support Windows, even if no one seemingly cares (I do, you have my support!).
Creating a header-only version
Ok, so we may decide that this whole static and dynamic library stuff isn’t for us. Perhaps you value your sanity and just can’t be bothered going through endless lines of compiler errors. Well, don’t give up on libraries just yet, there is something in store for you as well. When we discussed the differences between static, dynamic, and header-only libraries, we said that header-only libraries do not need to be compiled before usage, but instead contain their entire code in the interface, or header files. This means all we have to do is to include the required header files and we are done.
Even if this means that we have to change all of our code now, I want to show you how you could turn our linear algebra library into a header-only library if that is what you want to do. Since we do have to copy all source code into the header files, the easiest solution is to provide an additional directory within our library folder, as well as a separate header include file. The new project structure is given below.
root
├── build
├── linearAlgebraLib
│ ├── linearAlgebraLib.hpp
│ ├── headerOnlyLinearAlgebraLib.hpp // new
│ ├── src
│ │ ├── vector.(hpp|cpp)
│ │ ├── sparseMatrixCSR.(hpp|cpp)
│ │ ├── linearAlgebraSolverBase.(hpp|cpp)
│ │ └── conjugateGradient.(hpp|cpp)
│ └── headerOnly // new
│ ├── vector.hpp // new
│ ├── sparseMatrixCSR.hpp // new
│ ├── linearAlgebraSolverBase.hpp // new
│ └── conjugateGradient.hpp // new
├── main.cpp
├── buildAndRun.(sh|bat)
├── testHeaderOnlyLib.(sh|bat) // new
├── testStaticLib.(sh|bat)
└── testDynamicLib.(sh|bat)The updated headerOnlyLinearAlgebraLib.hpp file simply points to the header files in the linearAlgebraLib/headerOnly directory and is given as
// headerOnlyLinearAlgebraLib.hpp
#include "headerOnly/sparseMatrixCSR.hpp"
#include "headerOnly/vector.hpp"
#include "headerOnly/linearAlgebraSolverBase.hpp"
#include "headerOnly/conjugateGradient.hpp"
The remaining files in the headerOnly directory simply contain the interface (class structure) and all of the implementation. Let’s look at the conjugateGradient.hpp file this time as an example, its content is now given as
// conjugateGradient.hpp
#pragma once
#include <iostream>
#include "linearAlgebraLib/headerOnly/linearAlgebraSolverBase.hpp"
#include "linearAlgebraLib/headerOnly/sparseMatrixCSR.hpp"
#include "linearAlgebraLib/headerOnly/vector.hpp"
namespace linearAlgebraLib {
class ConjugateGradient : public LinearAlgebraSolverBase {
public:
ConjugateGradient(unsigned numberOfCells) : LinearAlgebraSolverBase(numberOfCells) { }
virtual Vector solve(unsigned maxIterations, double convergenceThreshold) final override {
Vector r0(_rightHandSide.size());
Vector r1(_rightHandSide.size());
Vector p0(_rightHandSide.size());
Vector p1(_rightHandSide.size());
Vector x(_rightHandSide.size());
auto &A = _coefficientMatrix;
auto &b = _rightHandSide;
double alpha = 0.0;
double beta = 0.0;
unsigned iteration = 0;
p1 = b - A * x;
r1 = b - A * x;
do {
r0 = r1;
p0 = p1;
alpha = (r0.transpose() * r0) / (p0.transpose() * (A * p0));
x = x + alpha * p0;
r1 = r0 - alpha * A * p0;
beta = (r1.transpose() * r1) / (r0.transpose() * r0);
p1 = r1 + beta * p0;
++iteration;
} while (iteration < maxIterations && r1.getL2Norm() > convergenceThreshold);
return x;
}
};
} // namespace linearAlgebraLibWhere we only declared the solve() function on line 16 before and then provided the implementation in a conjugateGradient.cpp file afterwards, we now have both the definition and implementation in the same file. This is all we have to do for all header files, at which point all code has migrated into the interface.
Since we have now two competing header files we can use for our library (the header-only version or the one we used for the static and dynamic libraries before), we have to modify our main.cpp file a bit. Thankfully, we know about the preprocessor magic now and can reuse it in the header file as well. Before, our main.cpp file started in the following way:
#include "linearAlgebraLib/linearAlgebraLib.hpp"
// solve the heat equation implicitly of the form dT/dt = gamma * d^2 T/ dx^2 over a domain L using the conjugate
// gradient methdod
// initial condition: 0 everywhere
// boundary condition: T(0) = 0, T(L) = 1
int main() {
// code ...
}But now, we are going to change line 1 so we have now
#ifdef HEADERONLYLIB
#include "linearAlgebraLib/headerOnlyLinearAlgebraLib.hpp"
#else
#include "linearAlgebraLib/linearAlgebraLib.hpp"
#endif
// solve the heat equation implicitly of the form dT/dt = gamma * d^2 T/ dx^2 over a domain L using the conjugate
// gradient methdod
// initial condition: 0 everywhere
// boundary condition: T(0) = 0, T(L) = 1
int main() {
// code ...
}We define a macro called HEADERONLYLIB. If this is defined during compilation, then we include the header-only version of the library, if it isn’t defined, we fall back on the static/dynamic library header. This means we end up with the following build scripts on UNIX, named testHeaderOnlyLib.sh:
#!/bin/bash
rm -rf build
mkdir -p build
# compile main function and link against header-only library
g++ -g -O0 -Wall -Wextra -I. -DHEADERONLYLIB -o build/headerOnlyLibExample main.cpp
# run
echo "Running header-only library example"
./build/headerOnlyLibExampleAnd, the corresponding build script on Windows, named testHeaderOnlyLib.bat becomes:
@echo off
del /q build
mkdir build
REM compile main function and link against header-only library
cl /nologo /Zi /EHsc /Od /I. /DHEADERONLYLIB /Fe:build\headerOnlyLibExample.exe main.cpp
REM remove object files
del *.obj
REM run
echo Running header-only library example
build\headerOnlyLibExample.exeBoth of these files are located in our root directory, next to the testDynamicLib.(sh|bat) and testStaticLib.(sh|bat). Notice the absence of the compilation requirement for the library code, since all is located in the header file, we only need to compile the main.cpp function, which is done on line 7. For both UNIX and Windows, we create a HEADERONLYLIB define through the -D and /D flag, respectively, which will import the correct header for us. Since we only have a single source file to compile, we can let the compiler directly create an executable. To do so, we have removed the -c and /c compiler flag, which instructs the compiler to create the object files instead of the executable directly.
We also used to have a statement to remove any build artefacts (object files) for our UNIX build script. Since we compile straight into an executable, we no longer need that, since no intermediary object files are created. On Windows, we kept it as the cl compiler will still create that intermediate object file and will dump it next to our source file, i.e. main.cpp. So we kept the del command but point it to the root directory this time, not the build directory.
So, there you have it, a header-only version of our library as well. These libraries are very popular for C++-only libraries that make liberal use of templates. If you want to write your library with lots of template usage, header-only libraries are the way to go. If you want to support other programming languages such as C, Fortran, or even Python, templates can still be used but complicate the library creation (we essentially have to specify ahead of time what possible types to expect for the template and then all versions of that will be created). If this is where you want to be heading, SWIG can help you with that.
Testing our libraries for correctness
Ok, we have come a long way. We have written the code, we have provided build scripts and we have written a test (our main.cpp file). At this point, we simply want to test all of our code with the build scripts. Running the testStaticLib.sh, testDynamicLib.sh, and testHeaderOnlyLib.sh on UNIX provides the following output:
./testStaticLib.sh
Running static library example
Simulation successful, final residual: 4.72557e-15./testDynamicLib.sh
Running dynamic library example
Simulation successful, final residual: 4.72557e-15./testHeaderOnlyLib.sh
Running header-only library example
Simulation successful, final residual: 4.72557e-15On Windows, we get the following output after running the testStaticLib.bat, testDynamicLib.bat, and testHeaderOnlyLib.bat files:
.\testStaticLib.bat
A subdirectory or file build already exists.
sparseMatrixCSR.cpp
vector.cpp
linearAlgebraSolverBase.cpp
conjugateGradient.cpp
main.cpp
Running static library example
Simulation successful, final residual: 4.72557e-15.\testDynamicLib.bat
A subdirectory or file build already exists.
sparseMatrixCSR.cpp
vector.cpp
linearAlgebraSolverBase.cpp
conjugateGradient.cpp
main.cpp
Creating library build\dynamicLinearAlgebra.lib and object build\dynamicLinearAlgebra.exp
Creating library build\dynamicLibExample.lib and object build\dynamicLibExample.exp
Running dynamic library example
Simulation successful, final residual: 4.72557e-15.\testHeaderOnlyLib.bat
A subdirectory or file build already exists.
main.cpp
Running header-only library example
Simulation successful, final residual: 4.72557e-15While we see some more verbose output on Windows, what I want to point out here is that regardless of the operating system or library type, we are always getting the same final residual, to the last digit. This is to be expected if we change the library type, as long as we stay on the same operating system but if you are getting different values for different operating systems, that can happen as well. The importance here is that regardless of the library type (static, dynamic, header-only) we get the same result. And with that, we are done, we have generated our own library that we can reuse in the future (and we will come back to it).
Summary
This article was again a somewhat longer write-up but I felt it was important to have everything in one place, and not to split it up over two or three sections. I hope you don’t mind. But what we have achieved here is to take our library code and compile that into either a static, dynamic, or header-only library.
It is not up to you to decide which type you want to support for your libraries, but let me give you some pointers that may help you decide which library type is best for you:
- Static libraries: Use static libraries for small-ish projects. The total size of your library is not going to be a problem and user won’t complain that their executable file is getting too large. They are the simplest to build and link against. Once linked, i.e. embedded into the executable, you won’t face any runtime issues and this user-friendliness can save you and your end-users a lot of headaches.
- Dynamic libraries: While static libraries are compiled into the executable, dynamic libraries are loaded at runtime only when needed. This means no need to embed them into your code and for larger software projects (we are talking 10000+ lines of code), dynamic libraries are probably what you want to go for. So, if you download your favourite open-source library which contains 100,000+ lines of code, you probably want to compile them as a dynamic library. Another advantage is that you can keep using your code without recompiling it, even when the dynamic library changes. Since the dynamic library is loaded at runtime, you can replace an older version with a newer version. As long as the interface doesn’t change, you don’t have to change your code and thus there is no need to recompile your code.
- Header-only libraries: These libraries are good for two types: Either you want to make heavy use of templates (then header-only libraries really are your only option) or you want to provide a very user-friendly library, i.e. one which is easy to include for others in their project. As long as the library is small in size, that is usually a good option, but keep in mind that header-only libraries can’t be compiled ahead of time. That means that every time you include a header-only library in your code, you have to compile the library every time you compile your code that includes it. This can significantly increase development time. It is not uncommon to jump from a few seconds to a few minutes in compilation time. You may tolerate it at first, but it can really frustrate you in the long run. I’m speaking from experience here.
So there you have it. By now, you should have a good understanding of the differences in library types and how to write your own library. I don’t want to stop here just yet, there is still a bit more to cover. In the remaining articles of this series, I want to look at how to work with already developed third-party CFD libraries and how you can bring those into your code. I hope you are looking forward to that and I’ll see you in the next article.
Tom-Robin Teschner is a senior lecturer in computational fluid dynamics and course director for the MSc in computational fluid dynamics and the MSc in aerospace computational engineering at Cranfield University.