Choosing the right programming language for CFD development

When you work on an existing CFD code, you can’t change the language and are stuck with whatever someone else decided before you would be a good choice (and I really hope it is not Fortran 77 in your case (brings back memories from my PhD days (sigh))). But if you are starting with a blank slate and can choose your language, you should not just jump on board what currently seems to be the most used language but rather consider what type of language you want to use. There are essentially three choices here; compiled languages, interpreted languages, and just-in-time compiled languages. In this post, we will look at all of them.

In this series

In this article

The importance of choosing the right programming language for computational simulations

When it comes to computational simulations in fields like computational fluid dynamics (CFD), choosing the right programming language is crucial. The language you select can significantly impact the performance, development process, and overall efficiency of your simulations. Therefore, understanding the nuances and differences among programming languages becomes paramount.

The world of programming offers a wide range of options, each with its own strengths and weaknesses. By carefully considering the specific requirements of your CFD applications, you can make an informed decision about which language to use. Factors such as computational requirements, development resources, project goals, and platform considerations must all be taken into account.

Differences in compiled, interpreted, and just-in-time (JIT) compiled languages

Before diving into the intricacies of programming languages for CFD, it is important to grasp the fundamental differences between compiled, interpreted, and just-in-time (JIT) compiled languages.

  1. Compiled languages: Compiled languages are those in which the source code is translated into machine code before execution. This conversion, performed by a compiler, results in an executable file that can be directly run on the target platform. Examples of popular compiled languages in the context of CFD include Fortran, C, and C++.
  2. Interpreted languages: Interpreted languages, on the other hand, execute the source code directly without prior compilation. Instead, an interpreter reads and executes the code line by line. Python and MATLAB are well-known interpreted languages that are widely used in the field of CFD.
  3. Just-in-Time (JIT) compiled languages: JIT compiled languages combine elements of both compiled and interpreted languages. They initially employ an interpreter to execute the code, but during runtime, they also perform dynamic compilation of certain code segments into machine code, optimizing performance. Just-in-time compilation can be particularly beneficial for computationally intensive tasks. Examples of JIT compiled languages used in CFD include C#, Julia, and Java (with JIT compilers like V8).

Understanding these distinctions is crucial because it affects various aspects of your CFD simulations, including performance, execution speed, portability, and development processes. By selecting the most suitable programming language based on your specific requirements, you can optimise your simulations for efficiency and productivity.

In the upcoming sections, we will look into compiled and interpreted languages, exploring their characteristics, advantages, and disadvantages for CFD applications. We will also investigate the role of just-in-time compilation and how it bridges the gap between compiled and interpreted languages, offering potential performance benefits for CFD applications.

Compiled Programming Languages

Compiled programming languages offer high performance and efficiency in CFD. Examples include Fortran, C, C++. They are the gold standard when it comes to execution speed, but may require recompilation for different platforms. Development and debugging processes can be more complex.

Definition and characteristics of compiled languages

Compiled languages are programming languages that undergo a process called compilation before execution. In this process, the source code is translated into machine code, specific to the target platform, by a compiler. The resulting compiled program can be directly executed, providing a high level of performance and efficiency. The primary characteristics of compiled languages include:

  1. Efficiency: Compiled languages are known for their high execution speed and efficiency. Since the code is transformed into machine code, which is directly understood by the computer’s hardware, it can take full advantage of system resources, resulting in faster execution times.
  2. Strongly Typed: Compiled languages often enforce strict data type checking. This means variables and data structures must be explicitly defined, allowing for better error detection during the compilation process.

In the field of computational fluid dynamics, several compiled programming languages have gained prominence due to their performance and computational capabilities. Some notable examples include:

  1. Fortran: Fortran (short for Formula Translation) has a long-standing history in scientific computing and CFD. Its efficient array handling and mathematical libraries make it well-suited for numerically intensive simulations. Most legacy codes still use Fortran and last time you checked the weather forecast, the results you saw likely came from a Fortran codebase. Example codes in Fortran are NASA’s CFL3D and FUN3D solvers.
  2. C and C++: C and its object-oriented extension, C++, are widely used in CFD due to their low-level control and excellent performance. They provide extensive libraries, allowing for efficient memory management and close interaction with system hardware. Newer CFD codes these days are predominantly written in C++ (OpenFOAM, SU2) and C (code_saturne).

Advantages and disadvantages of compiled languages

  1. Performance and execution speed: The main advantage of compiled languages is their high performance. By directly translating code into machine instructions, compiled languages can optimise computational tasks, resulting in faster execution times for CFD simulations.
  2. Portability and platform dependence: One of the challenges with compiled languages is their platform dependence. Since the compiled code is specific to the target platform, it may require recompilation or modifications to run on different operating systems (windows, macOS, UNIX (Linux)) or hardware architectures (32 bit vs 64 bit). However, this burden can be mitigated by employing platform-independent compilers and build systems.
  3. Development and debugging process: While compiled languages offer high performance, the development and debugging process can sometimes be more complex compared to interpreted languages. Compilation errors can occur during the build process, requiring careful attention to syntax, type checking, and linking. Debugging compiled code may also involve more effort due to lower-level access and potential optimisation complexities. Small code bases may compile fast, but it is not uncommon for larger codes to require minutes if not hours to compile, requiring further strategies to compile in chunks rather than the entire code base.

Despite these challenges, compiled languages remain popular choices in the field of CFD due to their performance benefits and the availability of optimised libraries for scientific computing. CFD simulations are computationally expensive and we always seem to need more computing resources than we have available. Performance is key and as a good CFD engineer you need to know at least one compiled language well. C++ is a popular choice these days and, while there is a steep learning curve, especially for the object-orientated part and its ever-growing complexity, knowing C++ is considered an asset within the industry. It usually ranks within the top 5 of the most popular and used programming languages and with its massive steering committee behind it, it is unlikely to change in the near future.

Interpreted Programming Languages

Interpreted programming languages, like Python and MATLAB, are widely used in CFD as well. They offer flexibility, portability, and ease of use. While they may have slower execution speeds, advancements like just-in-time (JIT) compilation can enhance performance.

Definition and characteristics of interpreted languages

Interpreted languages are programming languages that execute the source code directly without prior compilation. Instead of producing an executable file, an interpreter reads and processes the code line by line during runtime. This interpretive approach offers certain characteristics:

  1. Flexibility: Interpreted languages are known for their flexibility and dynamic nature. They allow for interactive development and support features like dynamic typing (a variable can change its type, from example a string to a float (number)) and runtime code modification, which can enhance productivity and ease of use.
  2. Portability: Interpreted languages are generally more portable than compiled languages. Since the source code is interpreted rather than compiled into platform-specific machine code, interpreted programs can be run on different platforms without the need for recompilation.

Several interpreted programming languages have found widespread use due to their ease of implementation and extensive libraries. Some prominent examples include:

  1. Python: Python is a versatile and widely adopted interpreted language in scientific computing and CFD. Its simplicity, readability, and extensive ecosystem of libraries (e.g., NumPy, SciPy) make it an ideal choice for prototyping, data analysis, and visualisation tasks in CFD. Most of Python’s scientific packages (e.g. NumPy) are implemented in a compiled language, offering similar speeds as compiled language and thus these packages can drastically boost the performance of the underlying Python code.
  2. MATLAB: MATLAB is a popular interpreted language widely used in various scientific and engineering fields, including CFD. It provides powerful matrix manipulation capabilities, rich visualisation tools, and numerous specialised toolboxes tailored for numerical computations. Its language syntax is akin to C programming, and it provides a rich user interface in which the MATLAB code can be typed.

Advantages and disadvantages of interpreted languages

  1. Performance and execution speed: Interpreted languages are generally slower than their compiled counterparts due to the need for runtime interpretation. As mentioned above, interpreted languages may offer performance boosts through packages and libraries that are implemented in a compiled language which removes some of the disadvantages of interpreted languages. However, this will induce a slight overhead and these interpreted languages are always slower than pure compiled languages.
  2. Portability and platform independence: Interpreted languages excel in portability. Since the code is interpreted rather than compiled into machine code, programs can be easily run on different platforms with minimal modifications, enhancing portability and reducing platform dependence.
  3. Development and debugging process: Interpreted languages often offer a more user-friendly development experience. They typically have simpler syntax, dynamic typing, and interactive environments, enabling rapid prototyping, exploration, and debugging. However, the absence of a compilation step may delay error detection until runtime.

Despite potential performance limitations, interpreted languages have gained popularity in CFD due to their ease of use, prototyping capabilities, and extensive libraries. Their flexibility and portability make them suitable for a wide range of applications.

This website exists to create a community of like-minded CFD enthusiasts and I’d love to start a discussion with you. If you would like to be part of it, sign up using the link below and you will receive my OpenFOAM quick reference guide, as well as my guide on Tools every CFD developer needs for free.

Join now

Just-in-Time (JIT) Compilation

Just-in-Time (JIT) compilation enhances performance in CFD by dynamically compiling code segments during runtime. It offers performance improvements, dynamic optimisation, but also incurs initialisation overhead. JIT compilation is a valuable tool for optimising CFD simulations in interpreted languages.

Explanation of JIT compilation and its role in programming languages

Just-in-Time (JIT) compilation is a technique that combines elements of both compiled and interpreted languages. It aims to improve the performance of interpreted languages by dynamically compiling certain portions of the code into machine code during runtime. This process bridges the gap between interpreted and compiled languages, offering performance benefits while retaining the flexibility of interpretation.

How JIT compilation works in practice

In practice, JIT compilation involves several steps:

  1. Initial interpretation: The source code is initially interpreted and executed by an interpreter.
  2. Profiling and identification: During execution, the JIT compiler profiles the code, identifying hotspots – sections of the code that are executed frequently and could benefit from compilation.
  3. Just-in-Time compilation: When a hotspot is identified, the corresponding code is dynamically compiled into machine code. This compiled version replaces the interpreted code, improving performance for subsequent executions of that code segment. Typically examples are loops that take a long time to complete, e.g. loops in time and space.
  4. Dynamic optimization: The JIT compiler can apply further optimisations to the compiled code based on runtime information. This includes inlining function calls, eliminating redundant computations, and optimising memory access patterns.

Examples of popular JIT compiled languages in CFD

CFD codes are mostly written using either compiled or interpreted languages. There are edge cases, though, where JIT compiled languages may be a good choice. The list below provides an overview of suitable candidates:

  1. Julia: Julia is a high-level, high-performance programming language specifically designed for numerical and scientific computing. With its built-in just-in-time (JIT) compilation, Julia offers fast execution speeds, dynamic typing, and a user-friendly syntax, making it ideal for CFD simulations.
  2. C#: C Sharp is a modern, object-oriented programming language developed by Microsoft. It combines the power of static typing with the flexibility of JIT compilation. C# offers a rich set of features, including memory management, strong integration with the .NET framework, and extensive community support, making it a viable choice for CFD applications requiring performance and productivity. If you want to use it to its fullest extent, though, you have to develop on Windows.
  3. Python: Wait, what? Didn’t you say Python was interpreted? Yes, it is, but its rich ecosystem means that we have JIT compilation available as well through packages like Numba or PyPy. These packages enable JIT compilation in Python, allowing us to enjoy the benefits of increased performance while still making use of all the other features Python has to offer.

Benefits and drawbacks of JIT compilation in CFD applications

  1. Performance improvements: JIT compilation can significantly enhance the performance of CFD applications written in interpreted languages. By selectively compiling hotspots, computationally intensive code segments can execute at speeds approaching that of compiled languages. This enables faster simulations and data analysis.
  2. Dynamic optimization: JIT compilers have the advantage of being able to adapt to runtime conditions. They can dynamically optimise the code based on profiling information, improving performance by tailoring the compiled code to the specific execution context. This can lead to efficient memory usage and better utilization of hardware resources.
  3. Overhead and initialization time: JIT compilation introduces overhead during the initial execution of a program since the compiler needs to profile and compile the code. This can result in increased initialization time compared to purely interpreted languages. However, subsequent runs of the compiled code do not suffer from this overhead, and the overall performance gains can outweigh the initial cost.

While JIT compilation offers significant benefits, it is essential to consider potential drawbacks:

  1. Memory consumption: JIT compilers generate compiled code at runtime, resulting in increased memory consumption compared to purely interpreted execution. This can be a consideration when working with large-scale CFD simulations that require efficient memory management.
  2. Warm-up period: JIT compilers typically require a warm-up period to identify and compile hotspots. Initially, the interpreted code may be slower until the JIT compiler has gathered sufficient profiling data to optimise performance. This warm-up period can impact the performance of short-lived programs or scenarios where quick execution is crucial.

Overall, JIT compilation can be a valuable tool for optimising performance in CFD applications written in interpreted languages. Its ability to dynamically compile and optimize code during runtime offers a balance between performance and flexibility. However, despite some of the advantages listed above, JIT-compilated languages are seldom used to write CFD solvers.

Summary

Understanding the trade-offs between compiled and interpreted languages is essential for making informed decisions in CFD applications. Compiled languages excel in performance and execution speed, making them ideal for computationally intensive simulations. Interpreted languages offer flexibility, ease of use, and rapid prototyping capabilities. Just-in-time compiled languages offer a path in between these languages, taking the burden away from the programmer to identify computational hotspots while offering the same flexibility as interpreted languages.

Ultimately, the choice is yours and will depend on your project. However, I am opinionated about this, and I strongly believe that you need to know how to use C++. Period. It is the single best investment you can make to improve your CV if you are serious about C++ development. In the next post, we will explore in more detail why C++ is so powerful for CFD development and why now really is the time to start learning it if you have not already done so!