Memory Pool is an optimization technique of allocating a specific amount of memory beforehand and handle all allocation and deallocation of memory from a concerned software system from this pre-allocated memory (which is known as Memory Pool).

This is an optimization technique as you can handle memory according to your software and tune it accordingly and need not rely on underlying software stack. It brings in up to 20% performance improvement on Standard Software Systems. With this article at OPENGENUS, you will get the complete idea of Memory Pool.

Advantages & Disadvantages of Memory Pool

Memory Pool is an optimization technique which is frequently used in all major Software Systems. The advantages of Memory Pool are:

• Allocating a single large memory takes significantly less time than multiple small chunks of memory (amounting to the same value).
• Using Memory Pool guarantees that the system will not run out of memory during execution.
• Memory Pool allows a program to restrict this memory usage and choose approaches which consume less memory when less memory is available in Memory Pool to stay within the memory usage limit.
• Avoids Memory fragmentation and use CPU cache hits effectively.

The disadvantages of using Memory Pool are:

• Memory Pool creates a layer of abstraction between the Software System and Memory allocation routines (like malloc).
• Programmers need to create their own interface to interact with memory calls like malloc, realloc and others.
• Programmers need to implement memory allocation and fitting algorithms to effectively use the memory available in Memory Pool

Sample API of Memory Pool in C++

The first step will be to implement our own version of malloc and free so that we can control allocation and deallocation and restrict it to our pre-allocated memory.

void *MemoryPool::malloc(const size_t &size)
void MemoryPool::free(void *ptr)

You can support templates to return custom memory pointers:

template <typename T>
T *MemoryPool::malloc();

template <typename T>
void MemoryPool::free(T *ptr);

Notice that we can returning a single pointer in case of malloc.

The limitation is that: We can grow our Memory Pool only when there is sequential memory available. As this is unlikely, we need to modify our API to return pointer to pointer so that we can expand our Memory Pool size dynamically.

Hence, the new API will be:

template <typename T>
T **MemoryPool::malloc();

Sample Implementation in C++

To implement Memory Pool, some points to consider are:

• Create a class named MemoryPool
• Allocate static memory as a private attribute. Note this needs to be a Dynamically Allocated memory for best approach.
• Define a function allocate(int size1) to assign memory of size1 size from the statch memory of MemoryPool. This should return a pointer to pointer to accomodate resizing of Memory Pool.
• Define function resize() to resize memory allocated to Memory Pool

Following is the sample implementation of Memory Pool in C++:

class MemoryPool
{
public:
    MemoryPool(): ptr(mem) 
    {
    }

    void* allocate(int mem_size)
    {
        // Check if space is available
        assert((ptr + mem_size) <= (mem + sizeof mem) && "Pool exhausted!");
        void* mem = ptr;
        ptr += mem_size;
        return mem;
    }

private:
    MemoryPool(const MemoryPool&);
    MemoryPool& operator=(const MemoryPool&);
    // Note this is the pre-allocated space in Memory Pool
    char mem[4096];
    // This is the pointer to the last element in Memory Pool
    char* ptr;
};

We can use our Memory Pool implementation as follows:

{
    MemoryPool pool;

    // Allocate an instance of `DataStructure` using memory pool
    DataStructure* data = new(pool.allocate(sizeof(DataStructure))) DataStructure;
    
    /*
    Use the memory
    */
    
    // Release the memory
    data->~DataStructure();
}

Softwares that use Memory Pool

All advanced Software Systems use Memory Pool internally so that they are use the available memory as per their need in any optimized way and is not restricted to any methodology of underlying software.

Some Software Systems that use Memory Pool are:

• OneDNN by Intel: This is a library of various components of Neural Networks which can be attached to other ML Libraries to accelerate the performance on Intel systems. It uses a Memory Pool of its own.

• TensorFlow by Intel: TensorFlow is the most popular Machine Learning Inference Library which uses Eigen Library internally for fundamental operations. It has its own Memory Pool implementation.

Conclusion: Should you use Memory Pool?

If you are building a production level software and the way memory is allocated and used has a significant impact on the performance of the software, then you should definitely consider adding a Memory Pool.

We suggest to develop a prototype of your software without memory pool support.

Once the prototype is approved or looks as expected, you may move forward and implement support of memory pool as an optimization. You can expect up to 20% performance improvement in general. The exact improvement depends on the memory usage of your software.