Avoiding page faults in a real-time application

A page fault occurs when an application reads or writes memory that is not committed to physical memory. It is impossible (or very hard) to predict when a page fault will happen so they are another source of non-determinism in computers.

Fortunately, there is a function that allows you to commit all memory for a process and lock it down so that it cannot cause a page fault. It is mlockall(2). These are its two flags:

You usually call mlockall(2) during the start up of the application with both flags set to lock all current and future memory mappings.

Tip

Note that MCL_FUTURE is not magic in that there will still be non-deterministic delay when allocating or freeing heap memory using malloc()/free() or mmap(). Such operations are best done at start up and not in the main control loops.

Memory allocated on the stack is trickier because it is done automatically and if you call a function that makes the stack deeper than before, you will encounter more memory management delays. A simple fix is to grow the stack to a size larger than you think you will ever need at start up. The code would look like this:

#define MAX_STACK (512*1024)
static void stack_grow (void)
{
  char dummy[MAX_STACK];
  memset(dummy, 0, MAX_STACK);
  return;
}

int main(int argc, char* argv[])
{
  [...]
  stack_grow ();
  mlockall(MCL_CURRENT | MCL_FUTURE);
  [...]

The stack_grow() function allocates a large variable on the stack and then zeroes it to force those pages of memory to be committed to this process.