[/ / Copyright (c) 2005-2012 Ion Gaztanaga / / Distributed under the Boost Software License, Version 1.0. (See accompanying / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) /] [library Boost.Interprocess [quickbook 1.5] [authors [Gaztanaga, Ion]] [copyright 2005-2015 Ion Gaztanaga] [id interprocess] [dirname interprocess] [purpose Interprocess communication utilities] [license Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at [@http://www.boost.org/LICENSE_1_0.txt]) ] ] [section:intro Introduction] [*Boost.Interprocess] simplifies the use of common interprocess communication and synchronization mechanisms and offers a wide range of them: * Shared memory. * Memory-mapped files. * Semaphores, mutexes, condition variables and upgradable mutex types to place them in shared memory and memory mapped files. * Named versions of those synchronization objects, similar to UNIX/Windows sem_open/CreateSemaphore API. * File locking. * Relative pointers. * Message queues. [*Boost.Interprocess] also offers higher-level interprocess mechanisms to allocate dynamically portions of a shared memory or a memory mapped file (in general, to allocate portions of a fixed size memory segment). Using these mechanisms, [*Boost.Interprocess] offers useful tools to construct C++ objects, including STL-like containers, in shared memory and memory mapped files: * Dynamic creation of anonymous and named objects in a shared memory or memory mapped file. * STL-like containers compatible with shared memory/memory-mapped files. * STL-like allocators ready for shared memory/memory-mapped files implementing several memory allocation patterns (like pooling). [section:introduction_building_interprocess Building Boost.Interprocess] There is no need to compile [*Boost.Interprocess], since it's a header only library. Just include your Boost header directory in your compiler include path. [*Boost.Interprocess] depends on [@http://www.boost.org/libs/date_time/ [*Boost.DateTime]], which needs separate compilation. However, the subset used by [*Boost.Interprocess] does not need any separate compilation so the user can define `BOOST_DATE_TIME_NO_LIB` to avoid Boost from trying to automatically link the [*Boost.DateTime]. In POSIX systems, [*Boost.Interprocess] uses pthread system calls to implement classes like mutexes, condition variables, etc... In some operating systems, these POSIX calls are implemented in separate libraries that are not automatically linked by the compiler. For example, in some Linux systems POSIX pthread functions are implemented in `librt.a` library, so you might need to add that library when linking an executable or shared library that uses [*Boost.Interprocess]. If you obtain linking errors related to those pthread functions, please revise your system's documentation to know which library implements them. [endsect] [section:tested_compilers Tested compilers] [*Boost.Interprocess] has been tested in the following compilers/platforms: * Visual C++ >= 7.1. * GCC >= 4.1. [warning GCC < 4.3 and MSVC < 9.0 are deprecated and will be removed in the next version.] [endsect] [endsect] [section:quick_guide Quick Guide for the Impatient] [section:qg_memory_pool Using shared memory as a pool of unnamed memory blocks] You can just allocate a portion of a shared memory segment, copy the message to that buffer, send the offset of that portion of shared memory to another process, and you are done. Let's see the example: [import ../example/doc_ipc_message.cpp] [doc_ipc_message] [endsect] [section:qg_named_interprocess Creating named shared memory objects] You want to create objects in a shared memory segment, giving a string name to them so that any other process can find, use and delete them from the segment when the objects are not needed anymore. Example: [import ../example/doc_named_alloc.cpp] [doc_named_alloc] [endsect] [section:qg_offset_ptr Using an offset smart pointer for shared memory] [*Boost.Interprocess] offers offset_ptr smart pointer family as an offset pointer that stores the distance between the address of the offset pointer itself and the address of the pointed object. When offset_ptr is placed in a shared memory segment, it can point safely objects stored in the same shared memory segment, even if the segment is mapped in different base addresses in different processes. This allows placing objects with pointer members in shared memory. For example, if we want to create a linked list in shared memory: [import ../example/doc_offset_ptr.cpp] [doc_offset_ptr] To help with basic data structures, [*Boost.Interprocess] offers containers like vector, list, map, so you can avoid these manual data structures just like with standard containers. [endsect] [section:qg_interprocess_container Creating vectors in shared memory] [*Boost.Interprocess] allows creating complex objects in shared memory and memory mapped files. For example, we can construct STL-like containers in shared memory. To do this, we just need to create a special (managed) shared memory segment, declare a [*Boost.Interprocess] allocator and construct the vector in shared memory just if it was any other object. The class that allows this complex structures in shared memory is called [classref boost::interprocess::managed_shared_memory] and it's easy to use. Just execute this example without arguments: [import ../example/doc_spawn_vector.cpp] [doc_spawn_vector] The parent process will create an special shared memory class that allows easy construction of many complex data structures associated with a name. The parent process executes the same program with an additional argument so the child process opens the shared memory and uses the vector and erases it. [endsect] [section:qg_interprocess_map Creating maps in shared memory] Just like a vector, [*Boost.Interprocess] allows creating maps in shared memory and memory mapped files. The only difference is that like standard associative containers, [*Boost.Interprocess]'s map needs also the comparison functor when an allocator is passed in the constructor: [import ../example/doc_map.cpp] [doc_map] For a more advanced example including containers of containers, see the section [link interprocess.allocators_containers.containers_explained.containers_of_containers Containers of containers]. [endsect] [endsect] [section:some_basic_explanations Some basic explanations] [section:processes_and_threads Processes And Threads] [*Boost.Interprocess] does not work only with processes but also with threads. [*Boost.Interprocess] synchronization mechanisms can synchronize threads from different processes, but also threads from the same process. [endsect] [section:sharing_information Sharing information between processes] In the traditional programming model an operating system has multiple processes running and each process has its own address space. To share information between processes we have several alternatives: * Two processes share information using a [*file]. To access to the data, each process uses the usual file read/write mechanisms. When updating/reading a file shared between processes, we need some sort of synchronization, to protect readers from writers. * Two processes share information that resides in the [*kernel] of the operating system. This is the case, for example, of traditional message queues. The synchronization is guaranteed by the operating system kernel. * Two processes can share a [*memory] region. This is the case of classical shared memory or memory mapped files. Once the processes set up the memory region, the processes can read/write the data like any other memory segment without calling the operating system's kernel. This also requires some kind of manual synchronization between processes. [endsect] [section:persistence Persistence Of Interprocess Mechanisms] One of the biggest issues with interprocess communication mechanisms is the lifetime of the interprocess communication mechanism. It's important to know when an interprocess communication mechanism disappears from the system. In [*Boost.Interprocess], we can have 3 types of persistence: * [*Process-persistence]: The mechanism lasts until all the processes that have opened the mechanism close it, exit or crash. * [*Kernel-persistence]: The mechanism exists until the kernel of the operating system reboots or the mechanism is explicitly deleted. * [*Filesystem-persistence]: The mechanism exists until the mechanism is explicitly deleted. Some native POSIX and Windows IPC mechanisms have different persistence so it's difficult to achieve portability between Windows and POSIX native mechanisms. [*Boost.Interprocess] classes have the following persistence: [table Boost.Interprocess Persistence Table [[Mechanism] [Persistence]] [[Shared memory] [Kernel or Filesystem]] [[Memory mapped file] [Filesystem]] [[Process-shared mutex types] [Process]] [[Process-shared semaphore] [Process]] [[Process-shared condition] [Process]] [[File lock] [Process]] [[Message queue] [Kernel or Filesystem]] [[Named mutex] [Kernel or Filesystem]] [[Named semaphore] [Kernel or Filesystem]] [[Named condition] [Kernel or Filesystem]] ] As you can see, [*Boost.Interprocess] defines some mechanisms with "Kernel or Filesystem" persistence. This is because POSIX allows this possibility to native interprocess communication implementations. One could, for example, implement shared memory using memory mapped files and obtain filesystem persistence (for example, there is no proper known way to emulate kernel persistence with a user library for Windows shared memory using native shared memory, or process persistence for POSIX shared memory, so the only portable way is to define "Kernel or Filesystem" persistence). [endsect] [section:names Names Of Interprocess Mechanisms] Some interprocess mechanisms are anonymous objects created in shared memory or memory-mapped files but other interprocess mechanisms need a name or identifier so that two unrelated processes can use the same interprocess mechanism object. Examples of this are shared memory, named mutexes and named semaphores (for example, native windows CreateMutex/CreateSemaphore API family). The name used to identify an interprocess mechanism is not portable, even between UNIX systems. For this reason, [*Boost.Interprocess] limits this name to a C++ variable identifier or keyword: *Starts with a letter, lowercase or uppercase, such as a letter from a to z or from A to Z. Examples: ['Sharedmemory, sharedmemory, sHaReDmEmOrY...] *Can include letters, underscore, or digits. Examples: ['shm1, shm2and3, ShM3plus4...] [endsect] [section:constructors_destructors_and_resource_lifetime Constructors, destructors and lifetime of Interprocess named resources] Named [*Boost.Interprocess] resources (shared memory, memory mapped files, named mutexes/conditions/semaphores) have kernel or filesystem persistency. This means that even if all processes that have opened those resources end, the resource will still be accessible to be opened again and the resource can only be destructed via an explicit call to their static member `remove` function. This behavior can be easily understood, since it's the same mechanism used by functions controlling file opening/creation/erasure: [table Boost.Interprocess-Filesystem Analogy [[Named Interprocess resource] [Corresponding std file] [Corresponding POSIX operation]] [[Constructor] [std::fstream constructor][open]] [[Destructor] [std::fstream destructor] [close]] [[Member `remove`] [None. `std::remove`] [unlink]] ] Now the correspondence between POSIX and Boost.Interprocess regarding shared memory and named semaphores: [table Boost.Interprocess-POSIX shared memory [[`shared_memory_object` operation] [POSIX operation]] [[Constructor] [shm_open]] [[Destructor] [close]] [[Member `remove`] [shm_unlink]] ] [table Boost.Interprocess-POSIX named semaphore [[`named_semaphore` operation] [POSIX operation]] [[Constructor] [sem_open]] [[Destructor] [close]] [[Member `remove`] [sem_unlink]] ] The most important property is that [*destructors of named resources don't remove the resource from the system], they only liberate resources allocated by the system for use by the process for the named resource. [*To remove the resource from the system the programmer must use `remove`]. [endsect] [section:permissions Permissions] Named resources offered by [*Boost.Interprocess] must cope with platform-dependant permission issues also present when creating files. If a programmer wants to shared shared memory, memory mapped files or named synchronization mechanisms (mutexes, semaphores, etc...) between users, it's necessary to specify those permissions. Sadly, traditional UNIX and Windows permissions are very different and [*Boost.Interprocess] does not try to standardize permissions, but does not ignore them. All named resource creation functions take an optional [classref boost::interprocess::permissions permissions object] that can be configured with platform-dependant permissions. Since each mechanism can be emulated through different mechanisms (a semaphore might be implement using mapped files or native semaphores) permissions types could vary when the implementation of a named resource changes (eg.: in Windows mutexes require `synchronize permissions`, but that's not the case of files). To avoid this, [*Boost.Interprocess] relies on file-like permissions, requiring file read-write-delete permissions to open named synchronization mechanisms (mutex, semaphores, etc.) and appropriate read or read-write-delete permissions for shared memory. This approach has two advantages: it's similar to the UNIX philosophy and the programmer does not need to know how the named resource is implemented. [endsect] [endsect] [section:sharedmemorybetweenprocesses Sharing memory between processes] [section:sharedmemory Shared memory] [section:shared_memory_what_is What is shared memory?] Shared memory is the fastest interprocess communication mechanism. The operating system maps a memory segment in the address space of several processes, so that several processes can read and write in that memory segment without calling operating system functions. However, we need some kind of synchronization between processes that read and write shared memory. Consider what happens when a server process wants to send an HTML file to a client process that resides in the same machine using network mechanisms: * The server must read the file to memory and pass it to the network functions, that copy that memory to the OS's internal memory. * The client uses the network functions to copy the data from the OS's internal memory to its own memory. As we can see, there are two copies, one from memory to the network and another one from the network to memory. And those copies are made using operating system calls that normally are expensive. Shared memory avoids this overhead, but we need to synchronize both processes: * The server maps a shared memory in its address space and also gets access to a synchronization mechanism. The server obtains exclusive access to the memory using the synchronization mechanism and copies the file to memory. * The client maps the shared memory in its address space. Waits until the server releases the exclusive access and uses the data. Using shared memory, we can avoid two data copies, but we have to synchronize the access to the shared memory segment. [endsect] [section:shared_memory_steps Creating memory segments that can be shared between processes] To use shared memory, we have to perform 2 basic steps: * Request to the operating system a memory segment that can be shared between processes. The user can create/destroy/open this memory using a [*shared memory object]: ['An object that represents memory that can be mapped concurrently into the address space of more than one process.]. * Associate a part of that memory or the whole memory with the address space of the calling process. The operating system looks for a big enough memory address range in the calling process' address space and marks that address range as an special range. Changes in that address range are automatically seen by other process that also have mapped the same shared memory object. Once the two steps have been successfully completed, the process can start writing to and reading from the address space to send to and receive data from other processes. Now, let's see how can we do this using [*Boost.Interprocess]: [endsect] [section:shared_memory_header Header] To manage shared memory, you just need to include the following header: [c++] #include [endsect] [section:shared_memory_creating_shared_memory_segments Creating shared memory segments] As we've mentioned we have to use the `shared_memory_object` class to create, open and destroy shared memory segments that can be mapped by several processes. We can specify the access mode of that shared memory object (read only or read-write), just as if it was a file: * Create a shared memory segment. Throws if already created: [c++] using boost::interprocess; shared_memory_object shm_obj (create_only //only create ,"shared_memory" //name ,read_write //read-write mode ); * To open or create a shared memory segment: [c++] using boost::interprocess; shared_memory_object shm_obj (open_or_create //open or create ,"shared_memory" //name ,read_only //read-only mode ); * To only open a shared memory segment. Throws if does not exist: [c++] using boost::interprocess; shared_memory_object shm_obj (open_only //only open ,"shared_memory" //name ,read_write //read-write mode ); When a shared memory object is created, its size is 0. To set the size of the shared memory, the user must use the `truncate` function call, in a shared memory that has been opened with read-write attributes: [c++] shm_obj.truncate(10000); As shared memory has kernel or filesystem persistence, the user must explicitly destroy it. The `remove` operation might fail returning false if the shared memory does not exist, the file is open or the file is still memory mapped by other processes: [c++] using boost::interprocess; shared_memory_object::remove("shared_memory"); For more details regarding `shared_memory_object` see the [classref boost::interprocess::shared_memory_object] class reference. [endsect] [section:shared_memory_mapping_shared_memory_segments Mapping Shared Memory Segments] Once created or opened, a process just has to map the shared memory object in the process' address space. The user can map the whole shared memory or just part of it. The mapping process is done using the `mapped_region` class. The class represents a memory region that has been mapped from a shared memory or from other devices that have also mapping capabilities (for example, files). A `mapped_region` can be created from any `memory_mappable` object and as you might imagine, `shared_memory_object` is a `memory_mappable` object: [c++] using boost::interprocess; std::size_t ShmSize = ... //Map the second half of the memory mapped_region region ( shm //Memory-mappable object , read_write //Access mode , ShmSize/2 //Offset from the beginning of shm , ShmSize-ShmSize/2 //Length of the region ); //Get the address of the region region.get_address(); //Get the size of the region region.get_size(); The user can specify the offset from the mappable object where the mapped region should start and the size of the mapped region. If no offset or size is specified, the whole mappable object (in this case, shared memory) is mapped. If the offset is specified, but not the size, the mapped region covers from the offset until the end of the mappable object. For more details regarding `mapped_region` see the [classref boost::interprocess::mapped_region] class reference. [endsect] [section:shared_memory_a_simple_example A Simple Example] Let's see a simple example of shared memory use. A server process creates a shared memory object, maps it and initializes all the bytes to a value. After that, a client process opens the shared memory, maps it, and checks that the data is correctly initialized: [import ../example/doc_shared_memory.cpp] [doc_shared_memory] [endsect] [section:emulation Emulation for systems without shared memory objects] [*Boost.Interprocess] provides portable shared memory in terms of POSIX semantics. Some operating systems don't support shared memory as defined by POSIX: * Windows operating systems provide shared memory using memory backed by the paging file but the lifetime semantics are different from the ones defined by POSIX (see [link interprocess.sharedmemorybetweenprocesses.sharedmemory.windows_shared_memory Native windows shared memory] section for more information). * Some UNIX systems don't fully support POSIX shared memory objects at all. In those platforms, shared memory is emulated with mapped files created in a "boost_interprocess" folder created in a temporary files directory. In Windows platforms, if "Common AppData" key is present in the registry, "boost_interprocess" folder is created in that directory (in XP usually "C:\Documents and Settings\All Users\Application Data" and in Vista "C:\ProgramData"). For Windows platforms without that registry key and Unix systems, shared memory is created in the system temporary files directory ("/tmp" or similar). Because of this emulation, shared memory has filesystem lifetime in some of those systems. [endsect] [section:removing Removing shared memory] [classref boost::interprocess::shared_memory_object shared_memory_object] provides a static `remove` function to remove a shared memory objects. This function [*can] fail if the shared memory objects does not exist or it's opened by another process. Note that this function is similar to the standard C `int remove(const char *path)` function. In UNIX systems, `shared_memory_object::remove` calls `shm_unlink`: * The function will remove the name of the shared memory object named by the string pointed to by name. * If one or more references to the shared memory object exist when is unlinked, the name will be removed before the function returns, but the removal of the memory object contents will be postponed until all open and map references to the shared memory object have been removed. * Even if the object continues to exist after the last function call, reuse of the name will subsequently cause the creation of a [classref boost::interprocess::shared_memory_object] instance to behave as if no shared memory object of this name exists (that is, trying to open an object with that name will fail and an object of the same name can be created again). In Windows operating systems, current version supports an usually acceptable emulation of the UNIX unlink behaviour: the file is renamed with a random name and marked as ['to be deleted when the last open handle is closed]. [endsect] [section:anonymous_shared_memory Anonymous shared memory for UNIX systems] Creating a shared memory segment and mapping it can be a bit tedious when several processes are involved. When processes are related via `fork()` operating system call in UNIX systems a simpler method is available using anonymous shared memory. This feature has been implemented in UNIX systems mapping the device `\dev\zero` or just using the `MAP_ANONYMOUS` in a POSIX conformant `mmap` system call. This feature is wrapped in [*Boost.Interprocess] using the `anonymous_shared_memory()` function, which returns a `mapped_region` object holding an anonymous shared memory segment that can be shared by related processes. Here is an example: [import ../example/doc_anonymous_shared_memory.cpp] [doc_anonymous_shared_memory] Once the segment is created, a `fork()` call can be used so that `region` is used to communicate two related processes. [endsect] [section:windows_shared_memory Native windows shared memory] Windows operating system also offers shared memory, but the lifetime of this shared memory is very different to kernel or filesystem lifetime. The shared memory is created backed by the pagefile and it's automatically destroyed when the last process attached to the shared memory is destroyed. Because of this reason, there is no effective way to simulate kernel or filesystem persistence using native windows shared memory and [*Boost.Interprocess] emulates shared memory using memory mapped files. This assures portability between POSIX and Windows operating systems. However, accessing native windows shared memory is a common request of [*Boost.Interprocess] users because they want to access to shared memory created with other process that don't use [*Boost.Interprocess]. In order to manage the native windows shared memory [*Boost.Interprocess] offers the [classref boost::interprocess::windows_shared_memory windows_shared_memory] class. Windows shared memory creation is a bit different from portable shared memory creation: the size of the segment must be specified when creating the object and can't be specified through `truncate` like with the shared memory object. Take in care that when the last process attached to a shared memory is destroyed [*the shared memory is destroyed] so there is [*no persistency] with native windows shared memory. Sharing memory between services and user applications is also different. To share memory between services and user applications the name of the shared memory must start with the global namespace prefix `"Global\\"`. This global namespace enables processes on multiple client sessions to communicate with a service application. The server component can create the shared memory in the global namespace. Then a client session can use the "Global\" prefix to open that memory. The creation of a shared memory object in the global namespace from a session other than session zero is a privileged operation. Let's repeat the same example presented for the portable shared memory object: A server process creates a shared memory object, maps it and initializes all the bytes to a value. After that, a client process opens the shared memory, maps it, and checks that the data is correctly initialized. Take in care that [*if the server exits before the client connects to the shared memory the client connection will fail], because the shared memory segment is destroyed when no proces is attached to the memory. This is the server process: [import ../example/doc_windows_shared_memory.cpp] [doc_windows_shared_memory] As we can see, native windows shared memory needs synchronization to make sure that the shared memory won't be destroyed before the client is launched. [endsect] [section:xsi_shared_memory XSI shared memory] In many UNIX systems, the OS offers another shared memory memory mechanism, XSI (X/Open System Interfaces) shared memory segments, also known as "System V" shared memory. This shared memory mechanism is quite popular and portable, and it's not based in file-mapping semantics, but it uses special functions (`shmget`, `shmat`, `shmdt`, `shmctl`...). Unlike POSIX shared memory segments, XSI shared memory segments are not identified by names but by 'keys' usually created with `ftok`. XSI shared memory segments have kernel lifetime and must be explicitly removed. XSI shared memory does not support copy-on-write and partial shared memory mapping but it supports anonymous shared memory. [*Boost.Interprocess] offers simple ([classref boost::interprocess::xsi_shared_memory xsi_shared_memory]) and managed ([classref boost::interprocess::managed_xsi_shared_memory managed_xsi_shared_memory]) shared memory classes to ease the use of XSI shared memory. It also wraps key creation with the simple [classref boost::interprocess::xsi_key xsi_key] class. Let's repeat the same example presented for the portable shared memory object: A server process creates a shared memory object, maps it and initializes all the bytes to a value. After that, a client process opens the shared memory, maps it, and checks that the data is correctly initialized. This is the server process: [import ../example/doc_xsi_shared_memory.cpp] [doc_xsi_shared_memory] [endsect] [endsect] [section:mapped_file Memory Mapped Files] [section:mapped_file_what_is What is a memory mapped file?] File mapping is the association of a file's contents with a portion of the address space of a process. The system creates a file mapping to associate the file and the address space of the process. A mapped region is the portion of address space that the process uses to access the file's contents. A single file mapping can have several mapped regions, so that the user can associate parts of the file with the address space of the process without mapping the entire file in the address space, since the file can be bigger than the whole address space of the process (a 9GB DVD image file in a usual 32 bit systems). Processes read from and write to the file using pointers, just like with dynamic memory. File mapping has the following advantages: * Uniform resource use. Files and memory can be treated using the same functions. * Automatic file data synchronization and cache from the OS. * Reuse of C++ utilities (STL containers, algorithms) in files. * Shared memory between two or more applications. * Allows efficient work with a large files, without mapping the whole file into memory * If several processes use the same file mapping to create mapped regions of a file, each process' views contain identical copies of the file on disk. File mapping is not only used for interprocess communication, it can be used also to simplify file usage, so the user does not need to use file-management functions to write the file. The user just writes data to the process memory, and the operating systems dumps the data to the file. When two processes map the same file in memory, the memory that one process writes is seen by another process, so memory mapped files can be used as an interprocess communication mechanism. We can say that memory-mapped files offer the same interprocess communication services as shared memory with the addition of filesystem persistence. However, as the operating system has to synchronize the file contents with the memory contents, memory-mapped files are not as fast as shared memory. [endsect] [section:mapped_file_steps Using mapped files] To use memory-mapped files, we have to perform 2 basic steps: * Create a mappable object that represent an already created file of the filesystem. This object will be used to create multiple mapped regions of the the file. * Associate the whole file or parts of the file with the address space of the calling process. The operating system looks for a big enough memory address range in the calling process' address space and marks that address range as an special range. Changes in that address range are automatically seen by other process that also have mapped the same file and those changes are also transferred to the disk automatically. Once the two steps have been successfully completed, the process can start writing to and reading from the address space to send to and receive data from other processes and synchronize the file's contents with the changes made to the mapped region. Now, let's see how can we do this using [*Boost.Interprocess]: [endsect] [section:mapped_file_header Header] To manage mapped files, you just need to include the following header: [c++] #include [endsect] [section:mapped_file_creating_file Creating a file mapping] First, we have to link a file's contents with the process' address space. To do this, we have to create a mappable object that represents that file. This is achieved in [*Boost.Interprocess] creating a `file_mapping` object: [c++] using boost::interprocess; file_mapping m_file ("/usr/home/file" //filename ,read_write //read-write mode ); Now we can use the newly created object to create mapped regions. For more details regarding this class see the [classref boost::interprocess::file_mapping] class reference. [endsect] [section:mapped_file_mapping_regions Mapping File's Contents In Memory] After creating a file mapping, a process just has to map the shared memory in the process' address space. The user can map the whole shared memory or just part of it. The mapping process is done using the `mapped_region` class. as we have said before The class represents a memory region that has been mapped from a shared memory or from other devices that have also mapping capabilities: [c++] using boost::interprocess; std::size_t FileSize = ... //Map the second half of the file mapped_region region ( m_file //Memory-mappable object , read_write //Access mode , FileSize/2 //Offset from the beginning of shm , FileSize-FileSize/2 //Length of the region ); //Get the address of the region region.get_address(); //Get the size of the region region.get_size(); The user can specify the offset from the file where the mapped region should start and the size of the mapped region. If no offset or size is specified, the whole file is mapped. If the offset is specified, but not the size, the mapped region covers from the offset until the end of the file. If several processes map the same file, and a process modifies a memory range from a mapped region that is also mapped by other process, the changes are inmedially visible to other processes. However, the file contents on disk are not updated immediately, since that would hurt performance (writing to disk is several times slower than writing to memory). If the user wants to make sure that file's contents have been updated, it can flush a range from the view to disk. When the function returns, the flushing process has started but there is no guarantee that all data has been written to disk: [c++] //Flush the whole region region.flush(); //Flush from an offset until the end of the region region.flush(offset); //Flush a memory range starting on an offset region.flush(offset, size); Remember that the offset is [*not] an offset on the file, but an offset in the mapped region. If a region covers the second half of a file and flushes the whole region, only the half of the file is guaranteed to have been flushed. For more details regarding `mapped_region` see the [classref boost::interprocess::mapped_region] class reference. [endsect] [section:mapped_file_a_simple_example A Simple Example] Let's reproduce the same example described in the shared memory section, using memory mapped files. A server process creates a shared memory segment, maps it and initializes all the bytes to a value. After that, a client process opens the shared memory, maps it, and checks that the data is correctly initialized:: [import ../example/doc_file_mapping.cpp] [doc_file_mapping] [endsect] [endsect] [section:mapped_region More About Mapped Regions] [section:mapped_region_one_class One Class To Rule Them All] As we have seen, both `shared_memory_object` and `file_mapping` objects can be used to create `mapped_region` objects. A mapped region created from a shared memory object or a file mapping are the same class and this has many advantages. One can, for example, mix in STL containers mapped regions from shared memory and memory mapped files. Libraries that only depend on mapped regions can be used to work with shared memory or memory mapped files without recompiling them. [endsect] [section:mapped_region_address_mapping Mapping Address In Several Processes] In the example we have seen, the file or shared memory contents are mapped to the address space of the process, but the address was chosen by the operating system. If several processes map the same file/shared memory, the mapping address will be surely different in each process. Since each process might have used its address space in a different way (allocation of more or less dynamic memory, for example), there is no guarantee that the file/shared memory is going to be mapped in the same address. If two processes map the same object in different addresses, this invalidates the use of pointers in that memory, since the pointer (which is an absolute address) would only make sense for the process that wrote it. The solution for this is to use offsets (distance) between objects instead of pointers: If two objects are placed in the same shared memory segment by one process, [*the address of each object will be different] in another process but [*the distance between them (in bytes) will be the same]. So the first advice when mapping shared memory and memory mapped files is to avoid using raw pointers, unless you know what you are doing. Use offsets between data or relative pointers to obtain pointer functionality when an object placed in a mapped region wants to point to an object placed in the same mapped region. [*Boost.Interprocess] offers a smart pointer called [classref boost::interprocess::offset_ptr] that can be safely placed in shared memory and that can be used to point to another object placed in the same shared memory / memory mapped file. [endsect] [section:mapped_region_fixed_address_mapping Fixed Address Mapping] The use of relative pointers is less efficient than using raw pointers, so if a user can succeed mapping the same file or shared memory object in the same address in two processes, using raw pointers can be a good idea. To map an object in a fixed address, the user can specify that address in the `mapped region`'s constructor: [c++] mapped_region region ( shm //Map shared memory , read_write //Map it as read-write , 0 //Map from offset 0 , 0 //Map until the end , (void*)0x3F000000 //Map it exactly there ); However, the user can't map the region in any address, even if the address is not being used. The offset parameter that marks the start of the mapping region is also limited. These limitations are explained in the next section. [endsect] [section:mapped_region_mapping_problems Mapping Offset And Address Limitations] As mentioned, the user can't map the memory mappable object at any address and it can specify the offset of the mappable object that is equivalent to the start of the mapping region to an arbitrary value. Most operating systems limit the mapping address and the offset of the mappable object to a multiple of a value called [*page size]. This is due to the fact that the [*operating system performs mapping operations over whole pages]. If fixed mapping address is used, ['offset] and ['address] parameters should be multiples of that value. This value is, typically, 4KB or 8KB for 32 bit operating systems. [c++] //These might fail because the offset is not a multiple of the page size //and we are using fixed address mapping mapped_region region1( shm //Map shared memory , read_write //Map it as read-write , 1 //Map from offset 1 , 1 //Map 1 byte , (void*)0x3F000000 //Aligned mapping address ); //These might fail because the address is not a multiple of the page size mapped_region region2( shm //Map shared memory , read_write //Map it as read-write , 0 //Map from offset 0 , 1 //Map 1 byte , (void*)0x3F000001 //Not aligned mapping address ); Since the operating system performs mapping operations over whole pages, specifying a mapping ['size] or ['offset] that are not multiple of the page size will waste more resources than necessary. If the user specifies the following 1 byte mapping: [c++] //Map one byte of the shared memory object. //A whole memory page will be used for this. mapped_region region ( shm //Map shared memory , read_write //Map it as read-write , 0 //Map from offset 0 , 1 //Map 1 byte ); The operating system will reserve a whole page that will not be reused by any other mapping so we are going to waste [*(page size - 1)] bytes. If we want to use efficiently operating system resources, we should create regions whose size is a multiple of [*page size] bytes. If the user specifies the following two mapped regions for a file with which has `2*page_size` bytes: //Map the first quarter of the file //This will use a whole page mapped_region region1( shm //Map shared memory , read_write //Map it as read-write , 0 //Map from offset 0 , page_size/2 //Map page_size/2 bytes ); //Map the rest of the file //This will use a 2 pages mapped_region region2( shm //Map shared memory , read_write //Map it as read-write , page_size/2 //Map from offset 0 , 3*page_size/2 //Map the rest of the shared memory ); In this example, a half of the page is wasted in the first mapping and another half is wasted in the second because the offset is not a multiple of the page size. The mapping with the minimum resource usage would be to map whole pages: //Map the whole first half: uses 1 page mapped_region region1( shm //Map shared memory , read_write //Map it as read-write , 0 //Map from offset 0 , page_size //Map a full page_size ); //Map the second half: uses 1 page mapped_region region2( shm //Map shared memory , read_write //Map it as read-write , page_size //Map from offset 0 , page_size //Map the rest ); How can we obtain the [*page size]? The `mapped_region` class has a static function that returns that value: [c++] //Obtain the page size of the system std::size_t page_size = mapped_region::get_page_size(); The operating system might also limit the number of mapped memory regions per process or per system. [endsect] [endsect] [section:mapped_region_object_limitations Limitations When Constructing Objects In Mapped Regions] When two processes create a mapped region of the same mappable object, two processes can communicate writing and reading that memory. A process could construct a C++ object in that memory so that the second process can use it. However, a mapped region shared by multiple processes, can't hold any C++ object, because not every class is ready to be a process-shared object, specially, if the mapped region is mapped in different address in each process. [section:offset_pointer Offset pointers instead of raw pointers] When placing objects in a mapped region and mapping that region in different address in every process, raw pointers are a problem since they are only valid for the process that placed them there. To solve this, [*Boost.Interprocess] offers a special smart pointer that can be used instead of a raw pointer. So user classes containing raw pointers (or Boost smart pointers, that internally own a raw pointer) can't be safely placed in a process shared mapped region. These pointers must be replaced with offset pointers, and these pointers must point only to objects placed in the same mapped region if you want to use these shared objects from different processes. Of course, a pointer placed in a mapped region shared between processes should only point to an object of that mapped region. Otherwise, the pointer would point to an address that it's only valid one process and other processes may crash when accessing to that address. [endsect] [section:references_forbidden References forbidden] References suffer from the same problem as pointers (mainly because they are implemented as pointers). However, it is not possible to create a fully workable smart reference currently in C++ (for example, `operator .()` can't be overloaded). Because of this, if the user wants to put an object in shared memory, the object can't have any (smart or not) reference as a member. References will only work if the mapped region is mapped in the same base address in all processes sharing a memory segment. Like pointers, a reference placed in a mapped region should only point to an object of that mapped region. [endsect] [section:virtuality_limitation Virtuality forbidden] The virtual table pointer and the virtual table are in the address space of the process that constructs the object, so if we place a class with a virtual function or virtual base class, the virtual pointer placed in shared memory will be invalid for other processes and they will crash. This problem is very difficult to solve, since each process needs a different virtual table pointer and the object that contains that pointer is shared across many processes. Even if we map the mapped region in the same address in every process, the virtual table can be in a different address in every process. To enable virtual functions for objects shared between processes, deep compiler changes are needed and virtual functions would suffer a performance hit. That's why [*Boost.Interprocess] does not have any plan to support virtual function and virtual inheritance in mapped regions shared between processes. [endsect] [section:statics_warning Be careful with static class members] Static members of classes are global objects shared by all instances of the class. Because of this, static members are implemented as global variables in processes. When constructing a class with static members, each process has its own copy of the static member, so updating a static member in one process does not change the value of the static member the another process. So be careful with these classes. Static members are not dangerous if they are just constant variables initialized when the process starts, but they don't change at all (for example, when used like enums) and their value is the same for all processes. [endsect] [endsect] [endsect] [section:offset_ptr Mapping Address Independent Pointer: offset_ptr] When creating shared memory and memory mapped files to communicate two processes the memory segment can be mapped in a different address in each process: [c++] #include // ... using boost::interprocess; //Open a shared memory segment shared_memory_object shm_obj (open_only //open or create ,"shared_memory" //name ,read_only //read-only mode ); //Map the whole shared memory mapped_region region ( shm //Memory-mappable object , read_write //Access mode ); //This address can be different in each process void *addr = region.get_address(); This makes the creation of complex objects in mapped regions difficult: a C++ class instance placed in a mapped region might have a pointer pointing to another object also placed in the mapped region. Since the pointer stores an absolute address, that address is only valid for the process that placed the object there unless all processes map the mapped region in the same address. To be able to simulate pointers in mapped regions, users must use [*offsets] (distance between objects) instead of absolute addresses. The offset between two objects in a mapped region is the same for any process that maps the mapped region, even if that region is placed in different base addresses. To facilitate the use of offsets, [*Boost.Interprocess] offers [classref boost::interprocess::offset_ptr offset_ptr]. [classref boost::interprocess::offset_ptr offset_ptr] wraps all the background operations needed to offer a pointer-like interface. The class interface is inspired in Boost Smart Pointers and this smart pointer stores the offset (distance in bytes) between the pointee's address and it's own `this` pointer. Imagine a structure in a common 32 bit processor: [c++] struct structure { int integer1; //The compiler places this at offset 0 in the structure offset_ptr ptr; //The compiler places this at offset 4 in the structure int integer2; //The compiler places this at offset 8 in the structure }; //... structure s; //Assign the address of "integer1" to "ptr". //"ptr" will store internally "-4": // (char*)&s.integer1 - (char*)&s.ptr; s.ptr = &s.integer1; //Assign the address of "integer2" to "ptr". //"ptr" will store internally "4": // (char*)&s.integer2 - (char*)&s.ptr; s.ptr = &s.integer2; One of the big problems of `offset_ptr` is the representation of the null pointer. The null pointer can't be safely represented like an offset, since the absolute address 0 is always outside of the mapped region. Due to the fact that the segment can be mapped in a different base address in each process the distance between the address 0 and `offset_ptr` is different for every process. Some implementations choose the offset 0 (that is, an `offset_ptr` pointing to itself) as the null pointer pointer representation but this is not valid for many use cases since many times structures like linked lists or nodes from STL containers point to themselves (the end node in an empty container, for example) and 0 offset value is needed. An alternative is to store, in addition to the offset, a boolean to indicate if the pointer is null. However, this increments the size of the pointer and hurts performance. In consequence, [classref boost::interprocess::offset_ptr offset_ptr] defines offset 1 as the null pointer, meaning that this class [*can't] point to the byte after its own ['this] pointer: [c++] using namespace boost::interprocess; offset_ptr ptr; //Pointing to the next byte of it's own address //marks the smart pointer as null. ptr = (char*)&ptr + 1; //ptr is equal to null assert(!ptr); //This is the same as assigning the null value... ptr = 0; //ptr is also equal to null assert(!ptr); In practice, this limitation is not important, since a user almost never wants to point to this address. [classref boost::interprocess::offset_ptr offset_ptr] offers all pointer-like operations and random_access_iterator typedefs, so it can be used in STL algorithms requiring random access iterators and detected via traits. For more information about the members and operations of the class, see [classref boost::interprocess::offset_ptr offset_ptr reference]. [endsect] [section:synchronization_mechanisms Synchronization mechanisms] [section:synchronization_mechanisms_overview Synchronization mechanisms overview] As mentioned before, the ability to shared memory between processes through memory mapped files or shared memory objects is not very useful if the access to that memory can't be effectively synchronized. This is the same problem that happens with thread-synchronization mechanisms, where heap memory and global variables are shared between threads, but the access to these resources needs to be synchronized typically through mutex and condition variables. [*Boost.Threads] implements these synchronization utilities between threads inside the same process. [*Boost.Interprocess] implements similar mechanisms to synchronize threads from different processes. [section:synchronization_mechanisms_named_vs_anonymous Named And Anonymous Synchronization Mechanisms] [*Boost.Interprocess] presents two types of synchronization objects: * [*Named utilities]: When two processes want to create an object of such type, both processes must ['create] or ['open] an object using the same name. This is similar to creating or opening files: a process creates a file with using a `fstream` with the name ['filename] and another process opens that file using another `fstream` with the same ['filename] argument. [*Each process uses a different object to access to the resource, but both processes are using the same underlying resource]. * [*Anonymous utilities]: Since these utilities have no name, two processes must share [*the same object] through shared memory or memory mapped files. This is similar to traditional thread synchronization objects: [*Both processes share the same object]. Unlike thread synchronization, where global variables and heap memory is shared between threads of the same process, sharing objects between two threads from different process can be only possible through mapped regions that map the same mappable resource (for example, shared memory or memory mapped files). Each type has it's own advantages and disadvantages: * Named utilities are easier to handle for simple synchronization tasks, since both process don't have to create a shared memory region and construct the synchronization mechanism there. * Anonymous utilities can be serialized to disk when using memory mapped objects obtaining automatic persistence of synchronization utilities. One could construct a synchronization utility in a memory mapped file, reboot the system, map the file again, and use the synchronization utility again without any problem. This can't be achieved with named synchronization utilities. The main interface difference between named and anonymous utilities are the constructors. Usually anonymous utilities have only one constructor, whereas the named utilities have several constructors whose first argument is a special type that requests creation, opening or opening or creation of the underlying resource: [c++] using namespace boost::interprocess; //Create the synchronization utility. If it previously //exists, throws an error NamedUtility(create_only, ...) //Open the synchronization utility. If it does not previously //exist, it's created. NamedUtility(open_or_create, ...) //Open the synchronization utility. If it does not previously //exist, throws an error. NamedUtility(open_only, ...) On the other hand the anonymous synchronization utility can only be created and the processes must synchronize using other mechanisms who creates the utility: [c++] using namespace boost::interprocess; //Create the synchronization utility. AnonymousUtility(...) [endsect] [section:synchronization_mechanisms_types Types Of Synchronization Mechanisms] Apart from its named/anonymous nature, [*Boost.Interprocess] presents the following synchronization utilities: * Mutexes (named and anonymous) * Condition variables (named and anonymous) * Semaphores (named and anonymous) * Upgradable mutexes * File locks [endsect] [endsect] [section:mutexes Mutexes] [section:mutexes_whats_a_mutex What's A Mutex?] ['Mutex] stands for [*mut]ual [*ex]clusion and it's the most basic form of synchronization between processes. Mutexes guarantee that only one thread can lock a given mutex. If a code section is surrounded by a mutex locking and unlocking, it's guaranteed that only a thread at a time executes that section of code. When that thread [*unlocks] the mutex, other threads can enter to that code region: [c++] //The mutex has been previously constructed lock_the_mutex(); //This code will be executed only by one thread //at a time. unlock_the_mutex(); A mutex can also be [*recursive] or [*non-recursive]: * Recursive mutexes can be locked several times by the same thread. To fully unlock the mutex, the thread has to unlock the mutex the same times it has locked it. * Non-recursive mutexes can't be locked several times by the same thread. If a mutex is locked twice by a thread, the result is undefined, it might throw an error or the thread could be blocked forever. [endsect] [section:mutexes_mutex_operations Mutex Operations] All the mutex types from [*Boost.Interprocess] implement the following operations: [blurb ['[*void lock()]]] [*Effects:] The calling thread tries to obtain ownership of the mutex, and if another thread has ownership of the mutex, it waits until it can obtain the ownership. If a thread takes ownership of the mutex the mutex must be unlocked by the same thread. If the mutex supports recursive locking, the mutex must be unlocked the same number of times it is locked. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool try_lock()]]] [*Effects:] The calling thread tries to obtain ownership of the mutex, and if another thread has ownership of the mutex returns immediately. If the mutex supports recursive locking, the mutex must be unlocked the same number of times it is locked. [*Returns:] If the thread acquires ownership of the mutex, returns true, if the another thread has ownership of the mutex, returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool timed_lock(const boost::posix_time::ptime &abs_time)]]] [*Effects:] The calling thread will try to obtain exclusive ownership of the mutex if it can do so in until the specified time is reached. If the mutex supports recursive locking, the mutex must be unlocked the same number of times it is locked. [*Returns:] If the thread acquires ownership of the mutex, returns true, if the timeout expires returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*void unlock()]]] [*Precondition:] The thread must have exclusive ownership of the mutex. [*Effects:] The calling thread releases the exclusive ownership of the mutex. If the mutex supports recursive locking, the mutex must be unlocked the same number of times it is locked. [*Throws:] An exception derived from *interprocess_exception* on error. [important `boost::posix_time::ptime` absolute time points used by Boost.Interprocess synchronization mechanisms are UTC time points, not local time points] [endsect] [section:mutexes_interprocess_mutexes Boost.Interprocess Mutex Types And Headers] Boost.Interprocess offers the following mutex types: [c++] #include * [classref boost::interprocess::interprocess_mutex interprocess_mutex]: A non-recursive, anonymous mutex that can be placed in shared memory or memory mapped files. [c++] #include * [classref boost::interprocess::interprocess_recursive_mutex interprocess_recursive_mutex]: A recursive, anonymous mutex that can be placed in shared memory or memory mapped files. [c++] #include * [classref boost::interprocess::named_mutex named_mutex]: A non-recursive, named mutex. [c++] #include * [classref boost::interprocess::named_recursive_mutex named_recursive_mutex]: A recursive, named mutex. [endsect] [section:mutexes_scoped_lock Scoped lock] It's very important to unlock a mutex after the process has read or written the data. This can be difficult when dealing with exceptions, so usually mutexes are used with a scoped lock, a class that can guarantee that a mutex will always be unlocked even when an exception occurs. To use a scoped lock just include: [c++] #include Basically, a scoped lock calls [*unlock()] in its destructor, and a mutex is always unlocked when an exception occurs. Scoped lock has many constructors to lock, try_lock, timed_lock a mutex or not to lock it at all. [c++] using namespace boost::interprocess; //Let's create any mutex type: MutexType mutex; { //This will lock the mutex scoped_lock lock(mutex); //Some code //The mutex will be unlocked here } { //This will try_lock the mutex scoped_lock lock(mutex, try_to_lock); //Check if the mutex has been successfully locked if(lock){ //Some code } //If the mutex was locked it will be unlocked } { boost::posix_time::ptime abs_time = ... //This will timed_lock the mutex scoped_lock lock(mutex, abs_time); //Check if the mutex has been successfully locked if(lock){ //Some code } //If the mutex was locked it will be unlocked } For more information, check the [classref boost::interprocess::scoped_lock scoped_lock's reference]. [important `boost::posix_time::ptime` absolute time points used by Boost.Interprocess synchronization mechanisms are UTC time points, not local time points] [endsect] [section:mutexes_anonymous_example Anonymous mutex example] Imagine that two processes need to write traces to a cyclic buffer built in shared memory. Each process needs to obtain exclusive access to the cyclic buffer, write the trace and continue. To protect the cyclic buffer, we can store a process shared mutex in the cyclic buffer. Each process will lock the mutex before writing the data and will write a flag when ends writing the traces (`doc_anonymous_mutex_shared_data.hpp` header): [import ../example/doc_anonymous_mutex_shared_data.hpp] [doc_anonymous_mutex_shared_data] This is the process main process. Creates the shared memory, constructs the cyclic buffer and start writing traces: [import ../example/comp_doc_anonymous_mutexA.cpp] [doc_anonymous_mutexA] The second process opens the shared memory, obtains access to the cyclic buffer and starts writing traces: [import ../example/comp_doc_anonymous_mutexB.cpp] [doc_anonymous_mutexB] As we can see, a mutex is useful to protect data but not to notify an event to another process. For this, we need a condition variable, as we will see in the next section. [endsect] [section:mutexes_named_example Named mutex example] Now imagine that two processes want to write a trace to a file. First they write their name, and after that they write the message. Since the operating system can interrupt a process in any moment we can mix parts of the messages of both processes, so we need a way to write the whole message to the file atomically. To achieve this, we can use a named mutex so that each process locks the mutex before writing: [import ../example/doc_named_mutex.cpp] [doc_named_mutex] [endsect] [endsect] [section:conditions Conditions] [section:conditions_whats_a_condition What's A Condition Variable?] In the previous example, a mutex is used to ['lock] but we can't use it to ['wait] efficiently until the condition to continue is met. A condition variable can do two things: * [*wait]: The thread is blocked until some other thread notifies that it can continue because the condition that lead to waiting has disappeared. * [*notify]: The thread sends a signal to one blocked thread or to all blocked threads to tell them that they the condition that provoked their wait has disappeared. Waiting in a condition variable is always associated with a mutex. The mutex must be locked prior to waiting on the condition. When waiting on the condition variable, the thread unlocks the mutex and waits [*atomically]. When the thread returns from a wait function (because of a signal or a timeout, for example) the mutex object is again locked. [endsect] [section:conditions_interprocess_conditions Boost.Interprocess Condition Types And Headers] Boost.Interprocess offers the following condition types: [c++] #include * [classref boost::interprocess::interprocess_condition interprocess_condition]: An anonymous condition variable that can be placed in shared memory or memory mapped files to be used with [classref boost::interprocess::interprocess_mutex]. [c++] #include * [classref boost::interprocess::interprocess_condition_any interprocess_condition_any]: An anonymous condition variable that can be placed in shared memory or memory mapped files to be used with any lock type. [c++] #include * [classref boost::interprocess::named_condition named_condition]: A named condition variable to be used with [classref boost::interprocess::named_mutex named_mutex]. [c++] #include * [classref boost::interprocess::named_condition named_condition]: A named condition variable to be used with any lock type. Named conditions are similar to anonymous conditions, but they are used in combination with named mutexes. Several times, we don't want to store synchronization objects with the synchronized data: * We want to change the synchronization method (from interprocess to intra-process, or without any synchronization) using the same data. Storing the process-shared anonymous synchronization with the synchronized data would forbid this. * We want to send the synchronized data through the network or any other communication method. Sending the process-shared synchronization objects wouldn't have any sense. [endsect] [section:conditions_anonymous_example Anonymous condition example] Imagine that a process that writes a trace to a simple shared memory buffer that another process prints one by one. The first process writes the trace and waits until the other process prints the data. To achieve this, we can use two condition variables: the first one is used to block the sender until the second process prints the message and the second one to block the receiver until the buffer has a trace to print. The shared memory trace buffer (doc_anonymous_condition_shared_data.hpp): [import ../example/doc_anonymous_condition_shared_data.hpp] [doc_anonymous_condition_shared_data] This is the process main process. Creates the shared memory, places there the buffer and starts writing messages one by one until it writes "last message" to indicate that there are no more messages to print: [import ../example/comp_doc_anonymous_conditionA.cpp] [doc_anonymous_conditionA] The second process opens the shared memory and prints each message until the "last message" message is received: [import ../example/comp_doc_anonymous_conditionB.cpp] [doc_anonymous_conditionB] With condition variables, a process can block if it can't continue the work, and when the conditions to continue are met another process can wake it. [endsect] [endsect] [section:semaphores Semaphores] [section:semaphores_whats_a_semaphores What's A Semaphore?] A semaphore is a synchronization mechanism between processes based in an internal count that offers two basic operations: * [*Wait]: Tests the value of the semaphore count, and waits if the value is less than or equal than 0. Otherwise, decrements the semaphore count. * [*Post]: Increments the semaphore count. If any process is blocked, one of those processes is awoken. If the initial semaphore count is initialized to 1, a [*Wait] operation is equivalent to a mutex locking and [*Post] is equivalent to a mutex unlocking. This type of semaphore is known as a [*binary semaphore]. Although semaphores can be used like mutexes, they have a unique feature: unlike mutexes, a [*Post] operation need not be executed by the same thread/process that executed the [*Wait] operation. [endsect] [section:semaphores_interprocess_semaphores Boost.Interprocess Semaphore Types And Headers] Boost.Interprocess offers the following semaphore types: [c++] #include * [classref boost::interprocess::interprocess_semaphore interprocess_semaphore]: An anonymous semaphore that can be placed in shared memory or memory mapped files. [c++] #include * [classref boost::interprocess::named_semaphore named_semaphore]: A named semaphore. [endsect] [section:semaphores_anonymous_example Anonymous semaphore example] We will implement an integer array in shared memory that will be used to transfer data from one process to another process. The first process will write some integers to the array and the process will block if the array is full. The second process will copy the transmitted data to its own buffer, blocking if there is no new data in the buffer. This is the shared integer array (doc_anonymous_semaphore_shared_data.hpp): [import ../example/doc_anonymous_semaphore_shared_data.hpp] [doc_anonymous_semaphore_shared_data] This is the process main process. Creates the shared memory, places there the integer array and starts integers one by one, blocking if the array is full: [import ../example/comp_doc_anonymous_semaphoreA.cpp] [doc_anonymous_semaphoreA] The second process opens the shared memory and copies the received integers to it's own buffer: [import ../example/comp_doc_anonymous_semaphoreB.cpp] [doc_anonymous_semaphoreB] The same interprocess communication can be achieved with a condition variables and mutexes, but for several synchronization patterns, a semaphore is more efficient than a mutex/condition combination. [endsect] [endsect] [section:sharable_upgradable_mutexes Sharable and Upgradable Mutexes] [section:upgradable_whats_a_mutex What's a Sharable and an Upgradable Mutex?] Sharable and upgradable mutex are special mutex types that offers more locking possibilities than a normal mutex. Sometimes, we can distinguish between [*reading] the data and [*modifying] the data. If just some threads need to modify the data, and a plain mutex is used to protect the data from concurrent access, concurrency is pretty limited: two threads that only read the data will be serialized instead of being executed concurrently. If we allow concurrent access to threads that just read the data but we avoid concurrent access between threads that read and modify or between threads that modify, we can increase performance. This is specially true in applications where data reading is more common than data modification and the synchronized data reading code needs some time to execute. With a sharable mutex we can acquire 2 lock types: * [*Exclusive lock]: Similar to a plain mutex. If a thread acquires an exclusive lock, no other thread can acquire any lock (exclusive or other) until the exclusive lock is released. If any thread other has any lock other than exclusive, a thread trying to acquire an exclusive lock will block. This lock will be acquired by threads that will modify the data. * [*Sharable lock]: If a thread acquires a sharable lock, other threads can't acquire the exclusive lock. If any thread has acquired the exclusive lock a thread trying to acquire a sharable lock will block. This locking is executed by threads that just need to read the data. With an upgradable mutex we can acquire previous locks plus a new upgradable lock: * [*Upgradable lock]: Acquiring an upgradable lock is similar to acquiring a [*privileged sharable lock]. If a thread acquires an upgradable lock, other threads can acquire a sharable lock. If any thread has acquired the exclusive or upgradable lock a thread trying to acquire an upgradable lock will block. A thread that has acquired an upgradable lock, is guaranteed to be able to acquire atomically an exclusive lock when other threads that have acquired a sharable lock release it. This is used for a thread that [*maybe] needs to modify the data, but usually just needs to read the data. This thread acquires the upgradable lock and other threads can acquire the sharable lock. If the upgradable thread reads the data and it has to modify it, the thread can be promoted to acquire the exclusive lock: when all sharable threads have released the sharable lock, the upgradable lock is atomically promoted to an exclusive lock. The newly promoted thread can modify the data and it can be sure that no other thread has modified it while doing the transition. [*Only 1 thread can acquire the upgradable (privileged reader) lock]. To sum up: [table Locking Possibilities for a Sharable Mutex [[If a thread has acquired the...] [Other threads can acquire...]] [[Sharable lock] [many sharable locks]] [[Exclusive lock] [no locks]] ] [table Locking Possibilities for an Upgradable Mutex [[If a thread has acquired the...] [Other threads can acquire...]] [[Sharable lock] [many sharable locks and 1 upgradable lock]] [[Upgradable lock] [many sharable locks]] [[Exclusive lock] [no locks]] ] [endsect] [section:upgradable_transitions Lock transitions for Upgradable Mutex] A sharable mutex has no option to change the acquired lock for another lock atomically. On the other hand, for an upgradable mutex, a thread that has acquired a lock can try to acquire another lock type atomically. All lock transitions are not guaranteed to succeed. Even if a transition is guaranteed to succeed, some transitions will block the thread waiting until other threads release the sharable locks. [*Atomically] means that no other thread will acquire an Upgradable or Exclusive lock in the transition, [*so data is guaranteed to remain unchanged]: [table Transition Possibilities for an Upgradable Mutex [[If a thread has acquired the...] [It can atomically release the previous lock and...]] [[Sharable lock] [try to obtain (not guaranteed) immediately the Exclusive lock if no other thread has exclusive or upgrable lock]] [[Sharable lock] [try to obtain (not guaranteed) immediately the Upgradable lock if no other thread has exclusive or upgrable lock]] [[Upgradable lock] [obtain the Exclusive lock when all sharable locks are released]] [[Upgradable lock] [obtain the Sharable lock immediately]] [[Exclusive lock] [obtain the Upgradable lock immediately]] [[Exclusive lock] [obtain the Sharable lock immediately]] ] As we can see, an upgradable mutex is a powerful synchronization utility that can improve the concurrency. However, if most of the time we have to modify the data, or the synchronized code section is very short, it's more efficient to use a plain mutex, since it has less overhead. Upgradable lock shines when the synchronized code section is bigger and there are more readers than modifiers. [endsect] [section:sharable_upgradable_mutexes_operations Upgradable Mutex Operations] All the upgradable mutex types from [*Boost.Interprocess] implement the following operations: [section:sharable_upgradable_mutexes_operations_exclusive Exclusive Locking (Sharable & Upgradable Mutexes)] [blurb ['[*void lock()]]] [*Effects:] The calling thread tries to obtain exclusive ownership of the mutex, and if another thread has any ownership of the mutex (exclusive or other), it waits until it can obtain the ownership. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool try_lock()]]] [*Effects:] The calling thread tries to acquire exclusive ownership of the mutex without waiting. If no other thread has any ownership of the mutex (exclusive or other) this succeeds. [*Returns:] If it can acquire exclusive ownership immediately returns true. If it has to wait, returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool timed_lock(const boost::posix_time::ptime &abs_time)]]] [*Effects:] The calling thread tries to acquire exclusive ownership of the mutex waiting if necessary until no other thread has any ownership of the mutex (exclusive or other) or abs_time is reached. [*Returns:] If acquires exclusive ownership, returns true. Otherwise returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*void unlock()]]] [*Precondition:] The thread must have exclusive ownership of the mutex. [*Effects:] The calling thread releases the exclusive ownership of the mutex. [*Throws:] An exception derived from *interprocess_exception* on error. [endsect] [section:sharable_upgradable_mutexes_operations_sharable Sharable Locking (Sharable & Upgradable Mutexes)] [blurb ['[*void lock_sharable()]]] [*Effects:] The calling thread tries to obtain sharable ownership of the mutex, and if another thread has exclusive ownership of the mutex, waits until it can obtain the ownership. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool try_lock_sharable()]]] [*Effects:] The calling thread tries to acquire sharable ownership of the mutex without waiting. If no other thread has exclusive ownership of the mutex this succeeds. [*Returns:] If it can acquire sharable ownership immediately returns true. If it has to wait, returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool timed_lock_sharable(const boost::posix_time::ptime &abs_time)]]] [*Effects:] The calling thread tries to acquire sharable ownership of the mutex waiting if necessary until no other thread has exclusive ownership of the mutex or abs_time is reached. [*Returns:] If acquires sharable ownership, returns true. Otherwise returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*void unlock_sharable()]]] [*Precondition:] The thread must have sharable ownership of the mutex. [*Effects:] The calling thread releases the sharable ownership of the mutex. [*Throws:] An exception derived from *interprocess_exception* on error. [endsect] [section:upgradable_mutexes_operations_upgradable Upgradable Locking (Upgradable Mutex only)] [blurb ['[*void lock_upgradable()]]] [*Effects:] The calling thread tries to obtain upgradable ownership of the mutex, and if another thread has exclusive or upgradable ownership of the mutex, waits until it can obtain the ownership. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool try_lock_upgradable()]]] [*Effects:] The calling thread tries to acquire upgradable ownership of the mutex without waiting. If no other thread has exclusive or upgradable ownership of the mutex this succeeds. [*Returns:] If it can acquire upgradable ownership immediately returns true. If it has to wait, returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool timed_lock_upgradable(const boost::posix_time::ptime &abs_time)]]] [*Effects:] The calling thread tries to acquire upgradable ownership of the mutex waiting if necessary until no other thread has exclusive ownership of the mutex or abs_time is reached. [*Returns:] If acquires upgradable ownership, returns true. Otherwise returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*void unlock_upgradable()]]] [*Precondition:] The thread must have upgradable ownership of the mutex. [*Effects:] The calling thread releases the upgradable ownership of the mutex. [*Throws:] An exception derived from *interprocess_exception* on error. [endsect] [section:upgradable_mutexes_operations_demotions Demotions (Upgradable Mutex only)] [blurb ['[*void unlock_and_lock_upgradable()]]] [*Precondition:] The thread must have exclusive ownership of the mutex. [*Effects:] The thread atomically releases exclusive ownership and acquires upgradable ownership. This operation is non-blocking. [*Throws:] An exception derived from *interprocess_exception* on error. [blurb ['[*void unlock_and_lock_sharable()]]] [*Precondition:] The thread must have exclusive ownership of the mutex. [*Effects:] The thread atomically releases exclusive ownership and acquires sharable ownership. This operation is non-blocking. [*Throws:] An exception derived from *interprocess_exception* on error. [blurb ['[*void unlock_upgradable_and_lock_sharable()]]] [*Precondition:] The thread must have upgradable ownership of the mutex. [*Effects:] The thread atomically releases upgradable ownership and acquires sharable ownership. This operation is non-blocking. [*Throws:] An exception derived from *interprocess_exception* on error. [endsect] [section:upgradable_mutexes_operations_promotions Promotions (Upgradable Mutex only)] [blurb ['[*void unlock_upgradable_and_lock()]]] [*Precondition:] The thread must have upgradable ownership of the mutex. [*Effects:] The thread atomically releases upgradable ownership and acquires exclusive ownership. This operation will block until all threads with sharable ownership release it. [*Throws:] An exception derived from *interprocess_exception* on error.[blurb ['[*bool try_unlock_upgradable_and_lock()]]] [*Precondition:] The thread must have upgradable ownership of the mutex. [*Effects:] The thread atomically releases upgradable ownership and tries to acquire exclusive ownership. This operation will fail if there are threads with sharable ownership, but it will maintain upgradable ownership. [*Returns:] If acquires exclusive ownership, returns true. Otherwise returns false. [*Throws:] An exception derived from *interprocess_exception* on error.[blurb ['[*bool timed_unlock_upgradable_and_lock(const boost::posix_time::ptime &abs_time)]]] [*Precondition:] The thread must have upgradable ownership of the mutex. [*Effects:] The thread atomically releases upgradable ownership and tries to acquire exclusive ownership, waiting if necessary until abs_time. This operation will fail if there are threads with sharable ownership or timeout reaches, but it will maintain upgradable ownership. [*Returns:] If acquires exclusive ownership, returns true. Otherwise returns false. [*Throws:] An exception derived from *interprocess_exception* on error.[blurb ['[*bool try_unlock_sharable_and_lock()]]] [*Precondition:] The thread must have sharable ownership of the mutex. [*Effects:] The thread atomically releases sharable ownership and tries to acquire exclusive ownership. This operation will fail if there are threads with sharable or upgradable ownership, but it will maintain sharable ownership. [*Returns:] If acquires exclusive ownership, returns true. Otherwise returns false. [*Throws:] An exception derived from *interprocess_exception* on error.[blurb ['[*bool try_unlock_sharable_and_lock_upgradable()]]] [*Precondition:] The thread must have sharable ownership of the mutex. [*Effects:] The thread atomically releases sharable ownership and tries to acquire upgradable ownership. This operation will fail if there are threads with sharable or upgradable ownership, but it will maintain sharable ownership. [*Returns:] If acquires upgradable ownership, returns true. Otherwise returns false. [*Throws:] An exception derived from *interprocess_exception* on error. [important `boost::posix_time::ptime` absolute time points used by Boost.Interprocess synchronization mechanisms are UTC time points, not local time points] [endsect] [endsect] [section:sharable_upgradable_mutexes_mutex_interprocess_mutexes Boost.Interprocess Sharable & Upgradable Mutex Types And Headers] Boost.Interprocess offers the following sharable mutex types: [c++] #include * [classref boost::interprocess::interprocess_sharable_mutex interprocess_sharable_mutex]: A non-recursive, anonymous sharable mutex that can be placed in shared memory or memory mapped files. [c++] #include * [classref boost::interprocess::named_sharable_mutex named_sharable_mutex]: A non-recursive, named sharable mutex. Boost.Interprocess offers the following upgradable mutex types: [c++] #include * [classref boost::interprocess::interprocess_upgradable_mutex interprocess_upgradable_mutex]: A non-recursive, anonymous upgradable mutex that can be placed in shared memory or memory mapped files. [c++] #include * [classref boost::interprocess::named_upgradable_mutex named_upgradable_mutex]: A non-recursive, named upgradable mutex. [endsect] [section:sharable_upgradable_locks Sharable Lock And Upgradable Lock] As with plain mutexes, it's important to release the acquired lock even in the presence of exceptions. [*Boost.Interprocess] mutexes are best used with the [classref boost::interprocess::scoped_lock scoped_lock] utility, and this class only offers exclusive locking. As we have sharable locking and upgradable locking with upgradable mutexes, we have two new utilities: [classref boost::interprocess::sharable_lock sharable_lock] and [classref boost::interprocess::upgradable_lock upgradable_lock]. Both classes are similar to `scoped_lock` but `sharable_lock` acquires the sharable lock in the constructor and `upgradable_lock` acquires the upgradable lock in the constructor. These two utilities can be use with any synchronization object that offers the needed operations. For example, a user defined mutex type with no upgradable locking features can use `sharable_lock` if the synchronization object offers [*lock_sharable()] and [*unlock_sharable()] operations: [section:upgradable_mutexes_lock_types Sharable Lock And Upgradable Lock Headers] [c++] #include [c++] #include [endsect] `sharable_lock` calls [*unlock_sharable()] in its destructor, and `upgradable_lock` calls [*unlock_upgradable()] in its destructor, so the upgradable mutex is always unlocked when an exception occurs. [c++] using namespace boost::interprocess; SharableOrUpgradableMutex sh_or_up_mutex; { //This will call lock_sharable() sharable_lock lock(sh_or_up_mutex); //Some code //The mutex will be unlocked here } { //This won't lock the mutex() sharable_lock lock(sh_or_up_mutex, defer_lock); //Lock it on demand. This will call lock_sharable() lock.lock(); //Some code //The mutex will be unlocked here } { //This will call try_lock_sharable() sharable_lock lock(sh_or_up_mutex, try_to_lock); //Check if the mutex has been successfully locked if(lock){ //Some code } //If the mutex was locked it will be unlocked } { boost::posix_time::ptime abs_time = ... //This will call timed_lock_sharable() scoped_lock lock(sh_or_up_mutex, abs_time); //Check if the mutex has been successfully locked if(lock){ //Some code } //If the mutex was locked it will be unlocked } UpgradableMutex up_mutex; { //This will call lock_upgradable() upgradable_lock lock(up_mutex); //Some code //The mutex will be unlocked here } { //This won't lock the mutex() upgradable_lock lock(up_mutex, defer_lock); //Lock it on demand. This will call lock_upgradable() lock.lock(); //Some code //The mutex will be unlocked here } { //This will call try_lock_upgradable() upgradable_lock lock(up_mutex, try_to_lock); //Check if the mutex has been successfully locked if(lock){ //Some code } //If the mutex was locked it will be unlocked } { boost::posix_time::ptime abs_time = ... //This will call timed_lock_upgradable() scoped_lock lock(up_mutex, abs_time); //Check if the mutex has been successfully locked if(lock){ //Some code } //If the mutex was locked it will be unlocked } [classref boost::interprocess::upgradable_lock upgradable_lock] and [classref boost::interprocess::sharable_lock sharable_lock] offer more features and operations, see their reference for more informations [important `boost::posix_time::ptime` absolute time points used by Boost.Interprocess synchronization mechanisms are UTC time points, not local time points] [endsect] [/section:upgradable_mutexes_example Anonymous Upgradable Mutex Example] [/import ../example/comp_doc_anonymous_upgradable_mutexA.cpp] [/doc_anonymous_upgradable_mutexA] [/import ../example/comp_doc_anonymous_upgradable_mutexB.cpp] [/doc_anonymous_upgradable_mutexB] [/endsect] [endsect] [section:lock_conversions Lock Transfers Through Move Semantics] [blurb [*Interprocess uses its own move semantics emulation code for compilers that don't support rvalues references. This is a temporary solution until a Boost move semantics library is accepted.]] Scoped locks and similar utilities offer simple resource management possibilities, but with advanced mutex types like upgradable mutexes, there are operations where an acquired lock type is released and another lock type is acquired atomically. This is implemented by upgradable mutex operations like `unlock_and_lock_sharable()`. These operations can be managed more effectively using [*lock transfer operations]. A lock transfer operations explicitly indicates that a mutex owned by a lock is transferred to another lock executing atomic unlocking plus locking operations. [section:lock_transfer_simple_transfer Simple Lock Transfer] Imagine that a thread modifies some data in the beginning but after that, it has to just read it in a long time. The code can acquire the exclusive lock, modify the data and atomically release the exclusive lock and acquire the sharable lock. With these sequence we guarantee that no other thread can modify the data in the transition and that more readers can acquire sharable lock, increasing concurrency. Without lock transfer operations, this would be coded like this: [c++] using boost::interprocess; interprocess_upgradable_mutex mutex; //Acquire exclusive lock mutex.lock(); //Modify data //Atomically release exclusive lock and acquire sharable lock. //More threads can acquire the sharable lock and read the data. mutex.unlock_and_lock_sharable(); //Read data //Explicit unlocking mutex.unlock_sharable(); This can be simple, but in the presence of exceptions, it's complicated to know what type of lock the mutex had when the exception was thrown and what function we should call to unlock it: [c++] try{ //Mutex operations } catch(...){ //What should we call? "unlock()" or "unlock_sharable()" //Is the mutex locked? } We can use [*lock transfer] to simplify all this management: [c++] using boost::interprocess; interprocess_upgradable_mutex mutex; //Acquire exclusive lock scoped_lock s_lock(mutex); //Modify data //Atomically release exclusive lock and acquire sharable lock. //More threads can acquire the sharable lock and read the data. sharable_lock(move(s_lock)); //Read data //The lock is automatically unlocked calling the appropriate unlock //function even in the presence of exceptions. //If the mutex was not locked, no function is called. As we can see, even if an exception is thrown at any moment, the mutex will be automatically unlocked calling the appropriate `unlock()` or `unlock_sharable()` method. [endsect] [section:lock_transfer_summary Lock Transfer Summary] There are many lock transfer operations that we can classify according to the operations presented in the upgradable mutex operations: * [*Guaranteed to succeed and non-blocking:] Any transition from a more restrictive lock to a less restrictive one. Scoped -> Upgradable, Scoped -> Sharable, Upgradable -> Sharable. * [*Not guaranteed to succeed:] The operation might succeed if no one has acquired the upgradable or exclusive lock: Sharable -> Exclusive. This operation is a try operation. * [*Guaranteed to succeed if using an infinite waiting:] Any transition that will succeed but needs to wait until all Sharable locks are released: Upgradable -> Scoped. Since this is a blocking operation, we can also choose not to wait infinitely and just try or wait until a timeout is reached. [section:lock_transfer_summary_scoped Transfers To Scoped Lock] Transfers to `scoped_lock` are guaranteed to succeed only from an `upgradable_lock` and only if a blocking operation is requested, due to the fact that this operation needs to wait until all sharable locks are released. The user can also use "try" or "timed" transfer to avoid infinite locking, but succeed is not guaranteed. A conversion from a `sharable_lock` is never guaranteed and thus, only a try operation is permitted: [c++] //Conversions to scoped_lock { upgradable_lock u_lock(mut); //This calls unlock_upgradable_and_lock() scoped_lock e_lock(move(u_lock)); } { upgradable_lock u_lock(mut); //This calls try_unlock_upgradable_and_lock() scoped_lock e_lock(move(u_lock, try_to_lock)); } { boost::posix_time::ptime t = test::delay(100); upgradable_lock u_lock(mut); //This calls timed_unlock_upgradable_and_lock() scoped_lock e_lock(move(u_lock)); } { sharable_lock s_lock(mut); //This calls try_unlock_sharable_and_lock() scoped_lock e_lock(move(s_lock, try_to_lock)); } [important `boost::posix_time::ptime` absolute time points used by Boost.Interprocess synchronization mechanisms are UTC time points, not local time points] [endsect] [section:lock_transfer_summary_upgradable Transfers To Upgradable Lock] A transfer to an `upgradable_lock` is guaranteed to succeed only from a `scoped_lock` since scoped locking is a more restrictive locking than an upgradable locking. This operation is also non-blocking. A transfer from a `sharable_lock` is not guaranteed and only a "try" operation is permitted: [c++] //Conversions to upgradable { sharable_lock s_lock(mut); //This calls try_unlock_sharable_and_lock_upgradable() upgradable_lock u_lock(move(s_lock, try_to_lock)); } { scoped_lock e_lock(mut); //This calls unlock_and_lock_upgradable() upgradable_lock u_lock(move(e_lock)); } [endsect] [section:lock_transfer_summary_sharable Transfers To Sharable Lock] All transfers to a `sharable_lock` are guaranteed to succeed since both `upgradable_lock` and `scoped_lock` are more restrictive than `sharable_lock`. These operations are also non-blocking: [c++] //Conversions to sharable_lock { upgradable_lock u_lock(mut); //This calls unlock_upgradable_and_lock_sharable() sharable_lock s_lock(move(u_lock)); } { scoped_lock e_lock(mut); //This calls unlock_and_lock_sharable() sharable_lock s_lock(move(e_lock)); } [endsect] [endsect] [section:lock_transfer_not_locked Transferring Unlocked Locks] In the previous examples, the mutex used in the transfer operation was previously locked: [c++] Mutex mut; //This calls mut.lock() scoped_lock e_lock(mut); //This calls unlock_and_lock_sharable() sharable_lock s_lock(move(e_lock)); } but it's possible to execute the transfer with an unlocked source, due to explicit unlocking, a try, timed or a `defer_lock` constructor: [c++] //These operations can leave the mutex unlocked! { //Try might fail scoped_lock e_lock(mut, try_to_lock); sharable_lock s_lock(move(e_lock)); } { //Timed operation might fail scoped_lock e_lock(mut, time); sharable_lock s_lock(move(e_lock)); } { //Avoid mutex locking scoped_lock e_lock(mut, defer_lock); sharable_lock s_lock(move(e_lock)); } { //Explicitly call unlock scoped_lock e_lock(mut); e_lock.unlock(); //Mutex was explicitly unlocked sharable_lock s_lock(move(e_lock)); } If the source mutex was not locked: * The target lock does not execute the atomic `unlock_xxx_and_lock_xxx` operation. * The target lock is also unlocked. * The source lock is released() and the ownership of the mutex is transferred to the target. [c++] { scoped_lock e_lock(mut, defer_lock); sharable_lock s_lock(move(e_lock)); //Assertions assert(e_lock.mutex() == 0); assert(s_lock.mutex() != 0); assert(e_lock.owns() == false); } [endsect] [section:lock_transfer_failure Transfer Failures] When executing a lock transfer, the operation can fail: * The executed atomic mutex unlock plus lock function might throw. * The executed atomic function might be a "try" or "timed" function that can fail. In the first case, the mutex ownership is not transferred and the source lock's destructor will unlock the mutex: [c++] { scoped_lock e_lock(mut, defer_lock); //This operations throws because //"unlock_and_lock_sharable()" throws!!! sharable_lock s_lock(move(e_lock)); //Some code ... //e_lock's destructor will call "unlock()" } In the second case, if an internal "try" or "timed" operation fails (returns "false") then the mutex ownership is [*not] transferred, the source lock is unchanged and the target lock's state will the same as a default construction: [c++] { sharable_lock s_lock(mut); //Internal "try_unlock_sharable_and_lock_upgradable()" returns false upgradable_lock u_lock(move(s_lock, try_to_lock)); assert(s_lock.mutex() == &mut); assert(s_lock.owns() == true); assert(u_lock.mutex() == 0); assert(u_lock.owns() == false); //u_lock's destructor does nothing //s_lock's destructor calls "unlock()" } [endsect] [endsect] [section:file_lock File Locks] [section:file_lock_whats_a_file_lock What's A File Lock?] A file lock is an interprocess synchronization mechanism to protect concurrent writes and reads to files using a mutex ['embedded] in the file. This ['embedded mutex] has sharable and exclusive locking capabilities. With a file lock, an existing file can be used as a mutex without the need of creating additional synchronization objects to control concurrent file reads or writes. Generally speaking, we can have two file locking capabilities: * [*Advisory locking:] The operating system kernel maintains a list of files that have been locked. But does not prevent writing to those files even if a process has acquired a sharable lock or does not prevent reading from the file when a process has acquired the exclusive lock. Any process can ignore an advisory lock. This means that advisory locks are for [*cooperating] processes, processes that can trust each other. This is similar to a mutex protecting data in a shared memory segment: any process connected to that memory can overwrite the data but [*cooperative] processes use mutexes to protect the data first acquiring the mutex lock. * [*Mandatory locking:] The OS kernel checks every read and write request to verify that the operation can be performed according to the acquired lock. Reads and writes block until the lock is released. [*Boost.Interprocess] implements [*advisory blocking] because of portability reasons. This means that every process accessing to a file concurrently, must cooperate using file locks to synchronize the access. In some systems file locking can be even further refined, leading to [*record locking], where a user can specify a [*byte range] within the file where the lock is applied. This allows concurrent write access by several processes if they need to access a different byte range in the file. [*Boost.Interprocess] does [*not] offer record locking for the moment, but might offer it in the future. To use a file lock just include: [c++] #include A file locking is a class that has [*process lifetime]. This means that if a process holding a file lock ends or crashes, the operating system will automatically unlock it. This feature is very useful in some situations where we want to assure automatic unlocking even when the process crashes and avoid leaving blocked resources in the system. A file lock is constructed using the name of the file as an argument: [c++] #include int main() { //This throws if the file does not exist or it can't //open it with read-write access! boost::interprocess::file_lock flock("my_file"); return 0; } [endsect] [section:file_lock_operations File Locking Operations] File locking has normal mutex operations plus sharable locking capabilities. This means that we can have multiple readers holding the sharable lock and writers holding the exclusive lock waiting until the readers end their job. However, file locking does [*not] support upgradable locking or promotion or demotion (lock transfers), so it's more limited than an upgradable lock. These are the operations: [blurb ['[*void lock()]]] [*Effects:] The calling thread tries to obtain exclusive ownership of the file lock, and if another thread has exclusive or sharable ownership of the mutex, it waits until it can obtain the ownership. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool try_lock()]]] [*Effects:] The calling thread tries to acquire exclusive ownership of the file lock without waiting. If no other thread has exclusive or sharable ownership of the file lock, this succeeds. [*Returns:] If it can acquire exclusive ownership immediately returns true. If it has to wait, returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool timed_lock(const boost::posix_time::ptime &abs_time)]]] [*Effects:] The calling thread tries to acquire exclusive ownership of the file lock waiting if necessary until no other thread has exclusive or sharable ownership of the file lock or abs_time is reached. [*Returns:] If acquires exclusive ownership, returns true. Otherwise returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*void unlock()]]] [*Precondition:] The thread must have exclusive ownership of the file lock. [*Effects:] The calling thread releases the exclusive ownership of the file lock. [*Throws:] An exception derived from *interprocess_exception* on error. [blurb ['[*void lock_sharable()]]] [*Effects:] The calling thread tries to obtain sharable ownership of the file lock, and if another thread has exclusive ownership of the file lock, waits until it can obtain the ownership. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool try_lock_sharable()]]] [*Effects:] The calling thread tries to acquire sharable ownership of the file lock without waiting. If no other thread has exclusive ownership of the file lock, this succeeds. [*Returns:] If it can acquire sharable ownership immediately returns true. If it has to wait, returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*bool timed_lock_sharable(const boost::posix_time::ptime &abs_time)]]] [*Effects:] The calling thread tries to acquire sharable ownership of the file lock waiting if necessary until no other thread has exclusive ownership of the file lock or abs_time is reached. [*Returns:] If acquires sharable ownership, returns true. Otherwise returns false. [*Throws:] *interprocess_exception* on error. [blurb ['[*void unlock_sharable()]]] [*Precondition:] The thread must have sharable ownership of the file lock. [*Effects:] The calling thread releases the sharable ownership of the file lock. [*Throws:] An exception derived from *interprocess_exception* on error. For more file locking methods, please [classref boost::interprocess::file_lock file_lock reference]. [important `boost::posix_time::ptime` absolute time points used by Boost.Interprocess synchronization mechanisms are UTC time points, not local time points] [endsect] [section:file_lock_scoped_locks Scoped Lock and Sharable Lock With File Locking] [classref boost::interprocess::scoped_lock scoped_lock] and [classref boost::interprocess::sharable_lock sharable_lock] can be used to make file locking easier in the presence of exceptions, just like with mutexes: [c++] #include #include //... using namespace boost::interprocess; //This process reads the file // ... //Open the file lock file_lock f_lock("my_file"); { //Construct a sharable lock with the file lock. //This will call "f_lock.sharable_lock()". sharable_lock sh_lock(f_lock); //Now read the file... //The sharable lock is automatically released by //sh_lock's destructor } [c++] #include #include //... using namespace boost::interprocess; //This process writes the file // ... //Open the file lock file_lock f_lock("my_file"); { //Construct an exclusive lock with the file lock. //This will call "f_lock.lock()". scoped_lock e_lock(f_lock); //Now write the file... //The exclusive lock is automatically released by //e_lock's destructor } However, lock transfers are only allowed between same type of locks, that is, from a sharable lock to another sharable lock or from a scoped lock to another scoped lock. A transfer from a scoped lock to a sharable lock is not allowed, because [classref boost::interprocess::file_lock file_lock] has no lock promotion or demotion functions like `unlock_and_lock_sharable()`. This will produce a compilation error: [c++] //Open the file lock file_lock f_lock("my_file"); scoped_lock e_lock(f_lock); //Compilation error, f_lock has no "unlock_and_lock_sharable()" member! sharable_lock e_lock(move(f_lock)); [endsect] [section:file_lock_not_thread_safe Caution: Synchronization limitations] If you plan to use file locks just like named mutexes, be careful, because portable file locks have synchronization limitations, mainly because different implementations (POSIX, Windows) offer different guarantees. Interprocess file locks have the following limitations: * It's unspecified if a `file_lock` synchronizes [*two threads from the same process]. * It's unspecified if a process can use two `file_lock` objects pointing to the same file. The first limitation comes mainly from POSIX, since a file handle is a per-process attribute and not a per-thread attribute. This means that if a thread uses a `file_lock` object to lock a file, other threads will see the file as locked. Windows file locking mechanism, on the other hand, offer thread-synchronization guarantees so a thread trying to lock the already locked file, would block. The second limitation comes from the fact that file locking synchronization state is tied with a single file descriptor in Windows. This means that if two `file_lock` objects are created pointing to the same file, no synchronization is guaranteed. In POSIX, when two file descriptors are used to lock a file if a descriptor is closed, all file locks set by the calling process are cleared. To sum up, if you plan to use portable file locking in your processes, use the following restrictions: * [*For each file, use a single `file_lock` object per process.] * [*Use the same thread to lock and unlock a file.] * If you are using a std::fstream/native file handle to write to the file while using file locks on that file, [*don't close the file before releasing all the locks of the file.] [endsect] [section:file_lock_careful_iostream Be Careful With Iostream Writing] As we've seen file locking can be useful to synchronize two processes, but [*make sure data is written to the file] before unlocking the file lock. Take in care that iostream classes do some kind of buffering, so if you want to make sure that other processes can see the data you've written, you have the following alternatives: * Use native file functions (read()/write() in Unix systems and ReadFile/WriteFile in Windows systems) instead of iostream. * Flush data before unlocking the file lock in writers using `fflush` if you are using standard C functions or the `flush()` member function when using C++ iostreams. In windows you can't even use another class to access the same file. //... using namespace boost::interprocess; //This process writes the file // ... //Open the file lock fstream file("my_file") file_lock f_lock("my_lock_file"); { scoped_lock e_lock(f_lock); //Now write the file... //Flush data before unlocking the exclusive lock file.flush(); } [endsect] [endsect] [section:message_queue Message Queue] [section:message_queue_whats_a_mq What's A Message Queue?] A message queue is similar to a list of messages. Threads can put messages in the queue and they can also remove messages from the queue. Each message can have also a [*priority] so that higher priority messages are read before lower priority messages. Each message has some attributes: * A priority. * The length of the message. * The data (if length is bigger than 0). A thread can send a message to or receive a message from the message queue using 3 methods: * [*Blocking]: If the message queue is full when sending or the message queue is empty when receiving, the thread is blocked until there is room for a new message or there is a new message. * [*Try]: If the message queue is full when sending or the message queue is empty when receiving, the thread returns immediately with an error. * [*Timed]: If the message queue is full when sending or the message queue is empty when receiving, the thread retries the operation until succeeds (returning successful state) or a timeout is reached (returning a failure). A message queue [*just copies raw bytes between processes] and does not send objects. This means that if we want to send an object using a message queue [*the object must be binary serializable]. For example, we can send integers between processes but [*not] a `std::string`. You should use [*Boost.Serialization] or use advanced [*Boost.Interprocess] mechanisms to send complex data between processes. The [*Boost.Interprocess] message queue is a named interprocess communication: the message queue is created with a name and it's opened with a name, just like a file. When creating a message queue, the user must specify the maximum message size and the maximum message number that the message queue can store. These parameters will define the resources (for example the size of the shared memory used to implement the message queue if shared memory is used). [c++] using boost::interprocess; //Create a message_queue. If the queue //exists throws an exception message_queue mq (create_only //only create ,"message_queue" //name ,100 //max message number ,100 //max message size ); [c++] using boost::interprocess; //Creates or opens a message_queue. If the queue //does not exist creates it, otherwise opens it. //Message number and size are ignored if the queue //is opened message_queue mq (open_or_create //open or create ,"message_queue" //name ,100 //max message number ,100 //max message size ); [c++] using boost::interprocess; //Opens a message_queue. If the queue //does not exist throws an exception. message_queue mq (open_only //only open ,"message_queue" //name ); The message queue is explicitly removed calling the static `remove` function: [c++] using boost::interprocess; message_queue::remove("message_queue"); The function can fail if the message queue is still being used by any process. [endsect] [section:message_queue_example Using a message queue] To use a message queue you must include the following header: [c++] #include In the following example, the first process creates the message queue, and writes an array of integers on it. The other process just reads the array and checks that the sequence number is correct. This is the first process: [import ../example/comp_doc_message_queueA.cpp] [doc_message_queueA] This is the second process: [import ../example/comp_doc_message_queueB.cpp] [doc_message_queueB] To know more about this class and all its operations, please see the [classref boost::interprocess::message_queue message_queue] class reference. [endsect] [endsect] [endsect] [section:managed_memory_segments Managed Memory Segments] [section:making_ipc_easy Making Interprocess Data Communication Easy] [section:managed_memory_segments_intro Introduction] As we have seen, [*Boost.Interprocess] offers some basic classes to create shared memory objects and file mappings and map those mappable classes to the process' address space. However, managing those memory segments is not not easy for non-trivial tasks. A mapped region is a fixed-length memory buffer and creating and destroying objects of any type dynamically, requires a lot of work, since it would require programming a memory management algorithm to allocate portions of that segment. Many times, we also want to associate names to objects created in shared memory, so all the processes can find the object using the name. [*Boost.Interprocess] offers 4 managed memory segment classes: * To manage a shared memory mapped region ([*basic_managed_shared_memory] class). * To manage a memory mapped file ([*basic_managed_mapped_file]). * To manage a heap allocated (`operator new`) memory buffer ([*basic_managed_heap_memory] class). * To manage a user provided fixed size buffer ([*basic_managed_external_buffer] class). The first two classes manage memory segments that can be shared between processes. The third is useful to create complex data-bases to be sent though other mechanisms like message queues to other processes. The fourth class can manage any fixed size memory buffer. The first two classes will be explained in the next two sections. [*basic_managed_heap_memory] and [*basic_managed_external_buffer] will be explained later. The most important services of a managed memory segment are: * Dynamic allocation of portions of a memory the segment. * Construction of C++ objects in the memory segment. These objects can be anonymous or we can associate a name to them. * Searching capabilities for named objects. * Customization of many features: memory allocation algorithm, index types or character types. * Atomic constructions and destructions so that if the segment is shared between two processes it's impossible to create two objects associated with the same name, simplifying synchronization. [endsect] [section:managed_memory_segment_int Declaration of managed memory segment classes] All [*Boost.Interprocess] managed memory segment classes are templatized classes that can be customized by the user: [c++] template < class CharType, class MemoryAlgorithm, template class IndexType > class basic_managed_shared_memory / basic_managed_mapped_file / basic_managed_heap_memory / basic_external_buffer; These classes can be customized with the following template parameters: * *CharType* is the type of the character that will be used to identify the created named objects (for example, *char* or *wchar_t*) * *MemoryAlgorithm* is the memory algorithm used to allocate portions of the segment (for example, rbtree_best_fit ). The internal typedefs of the memory algorithm also define: * The synchronization type (`MemoryAlgorithm::mutex_family`) to be used in all allocation operations. This allows the use of user-defined mutexes or avoiding internal locking (maybe code will be externally synchronized by the user). * The Pointer type (`MemoryAlgorithm::void_pointer`) to be used by the memory allocation algorithm or additional helper structures (like a map to maintain object/name associations). All STL compatible allocators and containers to be used with this managed memory segment will use this pointer type. The pointer type will define if the managed memory segment can be mapped between several processes. For example, if `void_pointer` is `offset_ptr` we will be able to map the managed segment in different base addresses in each process. If `void_pointer` is `void*` only fixed address mapping could be used. * See [link interprocess.customizing_interprocess.custom_interprocess_alloc Writing a new memory allocation algorithm] for more details about memory algorithms. * *IndexType* is the type of index that will be used to store the name-object association (for example, a map, a hash-map, or an ordered vector). This way, we can use `char` or `wchar_t` strings to identify created C++ objects in the memory segment, we can plug new shared memory allocation algorithms, and use the index type that is best suited to our needs. [endsect] [endsect] [section:managed_shared_memory Managed Shared Memory] [section:managed_memory_common_shm Common Managed Shared Memory Classes] As seen, *basic_managed_shared_memory* offers a great variety of customization. But for the average user, a common, default shared memory named object creation is needed. Because of this, [*Boost.Interprocess] defines the most common managed shared memory specializations: [c++] //!Defines a managed shared memory with c-strings as keys for named objects, //!the default memory algorithm (with process-shared mutexes, //!and offset_ptr as internal pointers) as memory allocation algorithm //!and the default index type as the index. //!This class allows the shared memory to be mapped in different base //!in different processes typedef basic_managed_shared_memory as void_pointer*/ ,/*Default index type*/> managed_shared_memory; //!Defines a managed shared memory with wide strings as keys for named objects, //!the default memory algorithm (with process-shared mutexes, //!and offset_ptr as internal pointers) as memory allocation algorithm //!and the default index type as the index. //!This class allows the shared memory to be mapped in different base //!in different processes typedef basic_managed_shared_memory as void_pointer*/ ,/*Default index type*/> wmanaged_shared_memory; `managed_shared_memory` allocates objects in shared memory associated with a c-string and `wmanaged_shared_memory` allocates objects in shared memory associated with a wchar_t null terminated string. Both define the pointer type as `offset_ptr` so they can be used to map the shared memory at different base addresses in different processes. If the user wants to map the shared memory in the same address in all processes and want to use raw pointers internally instead of offset pointers, [*Boost.Interprocess] defines the following types: [c++] //!Defines a managed shared memory with c-strings as keys for named objects, //!the default memory algorithm (with process-shared mutexes, //!and offset_ptr as internal pointers) as memory allocation algorithm //!and the default index type as the index. //!This class allows the shared memory to be mapped in different base //!in different processes*/ typedef basic_managed_shared_memory fixed_managed_shared_memory; //!Defines a managed shared memory with wide strings as keys for named objects, //!the default memory algorithm (with process-shared mutexes, //!and offset_ptr as internal pointers) as memory allocation algorithm //!and the default index type as the index. //!This class allows the shared memory to be mapped in different base //!in different processes typedef basic_managed_shared_memory wfixed_managed_shared_memory; [endsect] [section:constructing_managed_shared_memories Constructing Managed Shared Memory] Managed shared memory is an advanced class that combines a shared memory object and a mapped region that covers all the shared memory object. That means that when we [*create] a new managed shared memory: * A new shared memory object is created. * The whole shared memory object is mapped in the process' address space. * Some helper objects are constructed (name-object index, internal synchronization objects, internal variables...) in the mapped region to implement managed memory segment features. When we [*open] a managed shared memory * A shared memory object is opened. * The whole shared memory object is mapped in the process' address space. To use a managed shared memory, you must include the following header: [c++] #include [c++] //1. Creates a new shared memory object // called "MySharedMemory". //2. Maps the whole object to this // process' address space. //3. Constructs some objects in shared memory // to implement managed features. //!! If anything fails, throws interprocess_exception // managed_shared_memory segment ( create_only , "MySharedMemory" //Shared memory object name , 65536); //Shared memory object size in bytes [c++] //1. Opens a shared memory object // called "MySharedMemory". //2. Maps the whole object to this // process' address space. //3. Obtains pointers to constructed internal objects // to implement managed features. //!! If anything fails, throws interprocess_exception // managed_shared_memory segment (open_only, "MySharedMemory");//Shared memory object name [c++] //1. If the segment was previously created // equivalent to "open_only" (size is ignored). //2. Otherwise, equivalent to "create_only" //!! If anything fails, throws interprocess_exception // managed_shared_memory segment ( open_or_create , "MySharedMemory" //Shared memory object name , 65536); //Shared memory object size in bytes When the `managed_shared_memory` object is destroyed, the shared memory object is automatically unmapped, and all the resources are freed. To remove the shared memory object from the system you must use the `shared_memory_object::remove` function. Shared memory object removing might fail if any process still has the shared memory object mapped. The user can also map the managed shared memory in a fixed address. This option is essential when using using `fixed_managed_shared_memory`. To do this, just add the mapping address as an extra parameter: [c++] fixed_managed_shared_memory segment (open_only ,"MyFixedAddressSharedMemory" //Shared memory object name ,(void*)0x30000000 //Mapping address [endsect] [section:windows_managed_memory_common_shm Using native windows shared memory] Windows users might also want to use native windows shared memory instead of the portable [classref boost::interprocess::shared_memory_object shared_memory_object] managed memory. This is achieved through the [classref boost::interprocess::basic_managed_windows_shared_memory basic_managed_windows_shared_memory] class. To use it just include: [c++] #include This class has the same interface as [classref boost::interprocess::basic_managed_shared_memory basic_managed_shared_memory] but uses native windows shared memory. Note that this managed class has the same lifetime issues as the windows shared memory: when the last process attached to the windows shared memory is detached from the memory (or ends/crashes) the memory is destroyed. So there is no persistence support for windows shared memory. To communicate between system services and user applications using `managed_windows_shared_memory`, please read the explanations given in chapter [link interprocess.sharedmemorybetweenprocesses.sharedmemory.windows_shared_memory Native windows shared memory]. [endsect] [section:xsi_managed_memory_common_shm Using XSI (system V) shared memory] Unix users might also want to use XSI (system V) instead of the portable [classref boost::interprocess::shared_memory_object shared_memory_object] managed memory. This is achieved through the [classref boost::interprocess::basic_managed_xsi_shared_memory basic_managed_xsi_shared_memory] class. To use it just include: [c++] #include This class has nearly the same interface as [classref boost::interprocess::basic_managed_shared_memory basic_managed_shared_memory] but uses XSI shared memory as backend. [endsect] For more information about managed XSI shared memory capabilities, see [classref boost::interprocess::basic_managed_xsi_shared_memory basic_managed_xsi_shared_memory] class reference. [endsect] [section:managed_mapped_files Managed Mapped File] [section:managed_memory_common_mfile Common Managed Mapped Files] As seen, *basic_managed_mapped_file* offers a great variety of customization. But for the average user, a common, default shared memory named object creation is needed. Because of this, [*Boost.Interprocess] defines the most common managed mapped file specializations: [c++] //Named object creation managed memory segment //All objects are constructed in the memory-mapped file // Names are c-strings, // Default memory management algorithm(rbtree_best_fit with no mutexes) // Name-object mappings are stored in the default index type (flat_map) typedef basic_managed_mapped_file < char, rbtree_best_fit >, flat_map_index > managed_mapped_file; //Named object creation managed memory segment //All objects are constructed in the memory-mapped file // Names are wide-strings, // Default memory management algorithm(rbtree_best_fit with no mutexes) // Name-object mappings are stored in the default index type (flat_map) typedef basic_managed_mapped_file< wchar_t, rbtree_best_fit >, flat_map_index > wmanaged_mapped_file; `managed_mapped_file` allocates objects in a memory mapped files associated with a c-string and `wmanaged_mapped_file` allocates objects in a memory mapped file associated with a wchar_t null terminated string. Both define the pointer type as `offset_ptr` so they can be used to map the file at different base addresses in different processes. [endsect] [section:constructing_managed_mapped_files Constructing Managed Mapped Files] Managed mapped file is an advanced class that combines a file and a mapped region that covers all the file. That means that when we [*create] a new managed mapped file: * A new file is created. * The whole file is mapped in the process' address space. * Some helper objects are constructed (name-object index, internal synchronization objects, internal variables...) in the mapped region to implement managed memory segment features. When we [*open] a managed mapped file * A file is opened. * The whole file is mapped in the process' address space. To use a managed mapped file, you must include the following header: [c++] #include [c++] //1. Creates a new file // called "MyMappedFile". //2. Maps the whole file to this // process' address space. //3. Constructs some objects in the memory mapped // file to implement managed features. //!! If anything fails, throws interprocess_exception // managed_mapped_file mfile ( create_only , "MyMappedFile" //Mapped file name , 65536); //Mapped file size [c++] //1. Opens a file // called "MyMappedFile". //2. Maps the whole file to this // process' address space. //3. Obtains pointers to constructed internal objects // to implement managed features. //!! If anything fails, throws interprocess_exception // managed_mapped_file mfile ( open_only , "MyMappedFile"); //Mapped file name [c++] //1. If the file was previously created // equivalent to "open_only". //2. Otherwise, equivalent to "open_only" (size is ignored) // //!! If anything fails, throws interprocess_exception // managed_mapped_file mfile (open_or_create , "MyMappedFile" //Mapped file name , 65536); //Mapped file size When the `managed_mapped_file` object is destroyed, the file is automatically unmapped, and all the resources are freed. To remove the file from the filesystem you could use standard C `std::remove` or [*Boost.Filesystem]'s `remove()` functions, but file removing might fail if any process still has the file mapped in memory or the file is open by any process. To obtain a more portable behaviour, use `file_mapping::remove(const char *)` operation, which will remove the file even if it's being mapped. However, removal will fail in some OS systems if the file (eg. by C++ file streams) and no delete share permission was granted to the file. But in most common cases `file_mapping::remove` is portable enough. [endsect] For more information about managed mapped file capabilities, see [classref boost::interprocess::basic_managed_mapped_file basic_managed_mapped_file] class reference. [endsect] [section:managed_memory_segment_features Managed Memory Segment Features] The following features are common to all managed memory segment classes, but we will use managed shared memory in our examples. We can do the same with memory mapped files or other managed memory segment classes. [section:allocate_deallocate Allocating fragments of a managed memory segment] If a basic raw-byte allocation is needed from a managed memory segment, (for example, a managed shared memory), to implement top-level interprocess communications, this class offers [*allocate] and [*deallocate] functions. The allocation function comes with throwing and no throwing versions. Throwing version throws boost::interprocess::bad_alloc (which derives from `std::bad_alloc`) if there is no more memory and the non-throwing version returns 0 pointer. [import ../example/doc_managed_raw_allocation.cpp] [doc_managed_raw_allocation] [endsect] [section:segment_offset Obtaining handles to identify data] The class also offers conversions between absolute addresses that belong to a managed memory segment and a handle that can be passed using any interprocess mechanism. That handle can be transformed again to an absolute address using a managed memory segment that also contains that object. Handles can be used as keys between processes to identify allocated portions of a managed memory segment or objects constructed in the managed segment. [c++] //Process A obtains the offset of the address managed_shared_memory::handle handle = segment.get_handle_from_address(processA_address); //Process A sends this address using any mechanism to process B //Process B obtains the handle and transforms it again to an address managed_shared_memory::handle handle = ... void * processB_address = segment.get_address_from_handle(handle); [endsect] [section:allocation_types Object construction function family] When constructing objects in a managed memory segment (managed shared memory, managed mapped files...) associated with a name, the user has a varied object construction family to "construct" or to "construct if not found". [*Boost.Interprocess] can construct a single object or an array of objects. The array can be constructed with the same parameters for all objects or we can define each parameter from a list of iterators: [c++] //!Allocates and constructs an object of type MyType (throwing version) MyType *ptr = managed_memory_segment.construct("Name") (par1, par2...); //!Allocates and constructs an array of objects of type MyType (throwing version) //!Each object receives the same parameters (par1, par2, ...) MyType *ptr = managed_memory_segment.construct("Name")[count](par1, par2...); //!Tries to find a previously created object. If not present, allocates //!and constructs an object of type MyType (throwing version) MyType *ptr = managed_memory_segment.find_or_construct("Name") (par1, par2...); //!Tries to find a previously created object. If not present, allocates and //!constructs an array of objects of type MyType (throwing version). Each object //!receives the same parameters (par1, par2, ...) MyType *ptr = managed_memory_segment.find_or_construct("Name")[count](par1, par2...); //!Allocates and constructs an array of objects of type MyType (throwing version) //!Each object receives parameters returned with the expression (*it1++, *it2++,... ) MyType *ptr = managed_memory_segment.construct_it("Name")[count](it1, it2...); //!Tries to find a previously created object. If not present, allocates and constructs //!an array of objects of type MyType (throwing version). Each object receives //!parameters returned with the expression (*it1++, *it2++,... ) MyType *ptr = managed_memory_segment.find_or_construct_it("Name")[count](it1, it2...); //!Tries to find a previously created object. Returns a pointer to the object and the //!count (if it is not an array, returns 1). If not present, the returned pointer is 0 std::pair ret = managed_memory_segment.find("Name"); //!Destroys the created object, returns false if not present bool destroyed = managed_memory_segment.destroy("Name"); //!Destroys the created object via pointer managed_memory_segment.destroy_ptr(ptr); All these functions have a non-throwing version, that is invoked with an additional parameter std::nothrow. For example, for simple object construction: [c++] //!Allocates and constructs an object of type MyType (no throwing version) MyType *ptr = managed_memory_segment.construct("Name", std::nothrow) (par1, par2...); [endsect] [section:anonymous Anonymous instance construction] Sometimes, the user doesn't want to create class objects associated with a name. For this purpose, [*Boost.Interprocess] can create anonymous objects in a managed memory segment. All named object construction functions are available to construct anonymous objects. To allocate an anonymous objects, the user must use "boost::interprocess::anonymous_instance" name instead of a normal name: [c++] MyType *ptr = managed_memory_segment.construct(anonymous_instance) (par1, par2...); //Other construct variants can also be used (including non-throwing ones) ... //We can only destroy the anonymous object via pointer managed_memory_segment.destroy_ptr(ptr); Find functions have no sense here, since anonymous objects have no name. We can only destroy the anonymous object via pointer. [endsect] [section:unique Unique instance construction] Sometimes, the user wants to emulate a singleton in a managed memory segment. Obviously, as the managed memory segment is constructed at run-time, the user must construct and destroy this object explicitly. But how can the user be sure that the object is the only object of its type in the managed memory segment? This can be emulated using a named object and checking if it is present before trying to create one, but all processes must agree in the object's name, that can also conflict with other existing names. To solve this, [*Boost.Interprocess] offers a "unique object" creation in a managed memory segment. Only one instance of a class can be created in a managed memory segment using this "unique object" service (you can create more named objects of this class, though) so it makes easier the emulation of singleton-like objects across processes, for example, to design pooled, shared memory allocators. The object can be searched using the type of the class as a key. [c++] // Construct MyType *ptr = managed_memory_segment.construct(unique_instance) (par1, par2...); // Find it std::pair ret = managed_memory_segment.find(unique_instance); // Destroy it managed_memory_segment.destroy(unique_instance); // Other construct and find variants can also be used (including non-throwing ones) //... [c++] // We can also destroy the unique object via pointer MyType *ptr = managed_memory_segment.construct(unique_instance) (par1, par2...); managed_shared_memory.destroy_ptr(ptr); The find function obtains a pointer to the only object of type T that can be created using this "unique instance" mechanism. [endsect] [section:synchronization Synchronization guarantees] One of the features of named/unique allocations/searches/destructions is that they are [*atomic]. Named allocations use the recursive synchronization scheme defined by the internal `mutex_family` typedef defined of the memory allocation algorithm template parameter (`MemoryAlgorithm`). That is, the mutex type used to synchronize named/unique allocations is defined by the `MemoryAlgorithm::mutex_family::recursive_mutex_type` type. For shared memory, and memory mapped file based managed segments this recursive mutex is defined as [classref boost::interprocess::interprocess_recursive_mutex interprocess_recursive_mutex]. If two processes can call: [c++] MyType *ptr = managed_shared_memory.find_or_construct("Name")[count](par1, par2...); at the same time, but only one process will create the object and the other will obtain a pointer to the created object. Raw allocation using `allocate()` can be called also safely while executing named/anonymous/unique allocations, just like when programming a multithreaded application inserting an object in a mutex-protected map does not block other threads from calling new[] while the map thread is searching the place where it has to insert the new object. The synchronization does happen once the map finds the correct place and it has to allocate raw memory to construct the new value. This means that if we are creating or searching for a lot of named objects, we only block creation/searches from other processes but we don't block another process if that process is inserting elements in a shared memory vector. [endsect] [section:index_types Index types for name/object mappings] As seen, managed memory segments, when creating named objects, store the name/object association in an index. The index is a map with the name of the object as a key and a pointer to the object as the mapped type. The default specializations, *managed_shared_memory* and *wmanaged_shared_memory*, use *flat_map_index* as the index type. Each index has its own characteristics, like search-time, insertion time, deletion time, memory use, and memory allocation patterns. [*Boost.Interprocess] offers 3 index types right now: * [*boost::interprocess::flat_map_index flat_map_index]: Based on boost::interprocess::flat_map, an ordered vector similar to Loki library's AssocVector class, offers great search time and minimum memory use. But the vector must be reallocated when is full, so all data must be copied to the new buffer. Ideal when insertions are mainly in initialization time and in run-time we just need searches. * [*boost::interprocess::map_index map_index]: Based on boost::interprocess::map, a managed memory ready version of std::map. Since it's a node based container, it has no reallocations, the tree must be just rebalanced sometimes. Offers equilibrated insertion/deletion/search times with more overhead per node comparing to *boost::interprocess::flat_map_index*. Ideal when searches/insertions/deletions are in random order. * [*boost::interprocess::null_index null_index]: This index is for people using a managed memory segment just for raw memory buffer allocations and they don't make use of named/unique allocations. This class is just empty and saves some space and compilation time. If you try to use named object creation with a managed memory segment using this index, you will get a compilation error. As an example, if we want to define new managed shared memory class using *boost::interprocess::map* as the index type we just must specify [boost::interprocess::map_index map_index] as a template parameter: [c++] //This managed memory segment can allocate objects with: // -> a wchar_t string as key // -> boost::interprocess::rbtree_best_fit with process-shared mutexes // as memory allocation algorithm. // -> boost::interprocess::map<...> as the index to store name/object mappings // typedef boost::interprocess::basic_managed_shared_memory < wchar_t , boost::interprocess::rbtree_best_fit > , boost::interprocess::map_index > my_managed_shared_memory; [*Boost.Interprocess] plans to offer an *unordered_map* based index as soon as this container is included in Boost. If these indexes are not enough for you, you can define your own index type. To know how to do this, go to [link interprocess.customizing_interprocess.custom_indexes Building custom indexes] section. [endsect] [section:managed_memory_segment_segment_manager Segment Manager] All [*Boost.Interprocess] managed memory segment classes construct in their respective memory segments (shared memory, memory mapped files, heap memory...) some structures to implement the memory management algorithm, named allocations, synchronization objects... All these objects are encapsulated in a single object called [*segment manager]. A managed memory mapped file and a managed shared memory use the same [*segment manager] to implement all managed memory segment features, due to the fact that a [*segment manager] is a class that manages a fixed size memory buffer. Since both shared memory or memory mapped files are accessed though a mapped region, and a mapped region is a fixed size memory buffer, a single [*segment manager] class can manage several managed memory segment types. Some [*Boost.Interprocess] classes require a pointer to the segment manager in their constructors, and the segment manager can be obtained from any managed memory segment using `get_segment_manager` member: [c++] managed_shared_memory::segment_manager *seg_manager = managed_shm.get_segment_manager(); [endsect] [section:managed_memory_segment_information Obtaining information about a constructed object] Once an object is constructed using `construct<>` function family, the programmer can obtain information about the object using a pointer to the object. The programmer can obtain the following information: * Name of the object: If it's a named instance, the name used in the construction function is returned, otherwise 0 is returned. * Length of the object: Returns the number of elements of the object (1 if it's a single value, >=1 if it's an array). * The type of construction: Whether the object was constructed using a named, unique or anonymous construction. Here is an example showing this functionality: [import ../example/doc_managed_construction_info.cpp] [doc_managed_construction_info] [endsect] [section:managed_memory_segment_atomic_func Executing an object function atomically] Sometimes the programmer must execute some code, and needs to execute it with the guarantee that no other process or thread will create or destroy any named, unique or anonymous object while executing the functor. A user might want to create several named objects and initialize them, but those objects should be available for the rest of processes at once. To achieve this, the programmer can use the `atomic_func()` function offered by managed classes: [c++] //This object function will create several named objects create_several_objects_func func(/**/); //While executing the function, no other process will be //able to create or destroy objects managed_memory.atomic_func(func); Note that `atomic_func` does not prevent other processes from allocating raw memory or executing member functions for already constructed objects (e.g.: another process might be pushing elements into a vector placed in the segment). The atomic function only blocks named, unique and anonymous creation, search and destruction (concurrent calls to `construct<>`, `find<>`, `find_or_construct<>`, `destroy<>`...) from other processes. [endsect] [endsect] [section:managed_memory_segment_advanced_features Managed Memory Segment Advanced Features] [section:managed_memory_segment_information Obtaining information about the managed segment] These functions are available to obtain information about the managed memory segments: Obtain the size of the memory segment: [c++] managed_shm.get_size(); Obtain the number of free bytes of the segment: [c++] managed_shm.get_free_memory(); Clear to zero the free memory: [c++] managed_shm.zero_free_memory(); Know if all memory has been deallocated, false otherwise: [c++] managed_shm.all_memory_deallocated(); Test internal structures of the managed segment. Returns true if no errors are detected: [c++] managed_shm.check_sanity(); Obtain the number of named and unique objects allocated in the segment: [c++] managed_shm.get_num_named_objects(); managed_shm.get_num_unique_objects(); [endsect] [section:growing_managed_memory Growing managed segments] Once a managed segment is created the managed segment can't be grown. The limitation is not easily solvable: every process attached to the managed segment would need to be stopped, notified of the new size, they would need to remap the managed segment and continue working. Nearly impossible to achieve with a user-level library without the help of the operating system kernel. On the other hand, [*Boost.Interprocess] offers off-line segment growing. What does this mean? That the segment can be grown if no process has mapped the managed segment. If the application can find a moment where no process is attached it can grow or shrink to fit the managed segment. Here we have an example showing how to grow and shrink to fit [classref boost::interprocess::managed_shared_memory managed_shared_memory]: [import ../example/doc_managed_grow.cpp] [doc_managed_grow] [classref boost::interprocess::managed_mapped_file managed_mapped_file] also offers a similar function to grow or shrink_to_fit the managed file. Please, remember that [*no process should be modifying the file/shared memory while the growing/shrinking process is performed]. Otherwise, the managed segment will be corrupted. [endsect] [section:managed_memory_segment_advanced_index_functions Advanced index functions] As mentioned, the managed segment stores the information about named and unique objects in two indexes. Depending on the type of those indexes, the index must reallocate some auxiliary structures when new named or unique allocations are made. For some indexes, if the user knows how many named or unique objects are going to be created it's possible to preallocate some structures to obtain much better performance. (If the index is an ordered vector it can preallocate memory to avoid reallocations. If the index is a hash structure it can preallocate the bucket array). The following functions reserve memory to make the subsequent allocation of named or unique objects more efficient. These functions are only useful for pseudo-intrusive or non-node indexes (like `flat_map_index`, `iunordered_set_index`). These functions have no effect with the default index (`iset_index`) or other indexes (`map_index`): [c++] managed_shm.reserve_named_objects(1000); managed_shm.reserve_unique_objects(1000); [c++] managed_shm.reserve_named_objects(1000); managed_shm.reserve_unique_objects(1000); Managed memory segments also offer the possibility to iterate through constructed named and unique objects for debugging purposes. [*Caution: this iteration is not thread-safe] so the user should make sure that no other thread is manipulating named or unique indexes (creating, erasing, reserving...) in the segment. Other operations not involving indexes can be concurrently executed (raw memory allocation/deallocations, for example). The following functions return constant iterators to the range of named and unique objects stored in the managed segment. Depending on the index type, iterators might be invalidated after a named or unique creation/erasure/reserve operation: [c++] typedef managed_shared_memory::const_named_iterator const_named_it; const_named_it named_beg = managed_shm.named_begin(); const_named_it named_end = managed_shm.named_end(); typedef managed_shared_memory::const_unique_iterator const_unique_it; const_unique_it unique_beg = managed_shm.unique_begin(); const_unique_it unique_end = managed_shm.unique_end(); for(; named_beg != named_end; ++named_beg){ //A pointer to the name of the named object const managed_shared_memory::char_type *name = named_beg->name(); //The length of the name std::size_t name_len = named_beg->name_length(); //A constant void pointer to the named object const void *value = named_beg->value(); } for(; unique_beg != unique_end; ++unique_beg){ //The typeid(T).name() of the unique object const char *typeid_name = unique_beg->name(); //The length of the name std::size_t name_len = unique_beg->name_length(); //A constant void pointer to the unique object const void *value = unique_beg->value(); } [endsect] [section:allocate_aligned Allocating aligned memory portions] Sometimes it's interesting to be able to allocate aligned fragments of memory because of some hardware or software restrictions. Sometimes, having aligned memory is a feature that can be used to improve several memory algorithms. This allocation is similar to the previously shown raw memory allocation but it takes an additional parameter specifying the alignment. There is a restriction for the alignment: [*the alignment must be power of two]. If a user wants to allocate many aligned blocks (for example aligned to 128 bytes), the size that minimizes the memory waste is a value that's is nearly a multiple of that alignment (for example 2*128 - some bytes). The reason for this is that every memory allocation usually needs some additional metadata in the first bytes of the allocated buffer. If the user can know the value of "some bytes" and if the first bytes of a free block of memory are used to fulfill the aligned allocation, the rest of the block can be left also aligned and ready for the next aligned allocation. Note that requesting [*a size multiple of the alignment is not optimal] because lefts the next block of memory unaligned due to the needed metadata. Once the programmer knows the size of the payload of every memory allocation, he can request a size that will be optimal to allocate aligned chunks of memory maximizing both the size of the request [*and] the possibilities of future aligned allocations. This information is stored in the PayloadPerAllocation constant of managed memory segments. Here is a small example showing how aligned allocation is used: [import ../example/doc_managed_aligned_allocation.cpp] [doc_managed_aligned_allocation] [endsect] [section:managed_memory_segment_multiple_allocations Multiple allocation functions] [caution This feature is experimental, interface and ABI are unstable] If an application needs to allocate a lot of memory buffers but it needs to deallocate them independently, the application is normally forced to loop calling `allocate()`. Managed memory segments offer an alternative function to pack several allocations in a single call obtaining memory buffers that: * are packed contiguously in memory (which improves locality) * can be independently deallocated. This allocation method is much faster than calling `allocate()` in a loop. The downside is that the segment must provide a contiguous memory segment big enough to hold all the allocations. Managed memory segments offer this functionality through `allocate_many()` functions. There are 2 types of `allocate_many` functions: * Allocation of N buffers of memory with the same size. * Allocation of N buffers of memory, each one of different size. [c++] //!Allocates n_elements of elem_bytes bytes. //!Throws bad_alloc on failure. chain.size() is not increased on failure. void allocate_many(size_type elem_bytes, size_type n_elements, multiallocation_chain &chain); //!Allocates n_elements, each one of element_lengths[i]*sizeof_element bytes. //!Throws bad_alloc on failure. chain.size() is not increased on failure. void allocate_many(const size_type *element_lengths, size_type n_elements, size_type sizeof_element, multiallocation_chain &chain); //!Allocates n_elements of elem_bytes bytes. //!Non-throwing version. chain.size() is not increased on failure. void allocate_many(std::nothrow_t, size_type elem_bytes, size_type n_elements, multiallocation_chain &chain); //!Allocates n_elements, each one of //!element_lengths[i]*sizeof_element bytes. //!Non-throwing version. chain.size() is not increased on failure. void allocate_many(std::nothrow_t, const size_type *elem_sizes, size_type n_elements, size_type sizeof_element, multiallocation_chain &chain); //!Deallocates all elements contained in chain. //!Never throws. void deallocate_many(multiallocation_chain &chain); Here is a small example showing all this functionality: [import ../example/doc_managed_multiple_allocation.cpp] [doc_managed_multiple_allocation] Allocating N buffers of the same size improves the performance of pools and node containers (for example STL-like lists): when inserting a range of forward iterators in a STL-like list, the insertion function can detect the number of needed elements and allocate in a single call. The nodes still can be deallocated. Allocating N buffers of different sizes can be used to speed up allocation in cases where several objects must always be allocated at the same time but deallocated at different times. For example, a class might perform several initial allocations (some header data for a network packet, for example) in its constructor but also allocations of buffers that might be reallocated in the future (the data to be sent through the network). Instead of allocating all the data independently, the constructor might use `allocate_many()` to speed up the initialization, but it still can deallocate and expand the memory of the variable size element. In general, `allocate_many` is useful with large values of N. Overuse of `allocate_many` can increase the effective memory usage, because it can't reuse existing non-contiguous memory fragments that might be available for some of the elements. [endsect] [section:managed_memory_segment_expand_in_place Expand in place memory allocation] When programming some data structures such as vectors, memory reallocation becomes an important tool to improve performance. Managed memory segments offer an advanced reallocation function that offers: * Forward expansion: An allocated buffer can be expanded so that the end of the buffer is moved further. New data can be written between the old end and the new end. * Backwards expansion: An allocated buffer can be expanded so that the beginning of the buffer is moved backwards. New data can be written between the new beginning and the old beginning. * Shrinking: An allocated buffer can be shrunk so that the end of the buffer is moved backwards. The memory between the new end and the old end can be reused for future allocations. The expansion can be combined with the allocation of a new buffer if the expansion fails obtaining a function with "expand, if fails allocate a new buffer" semantics. Apart from this features, the function always returns the real size of the allocated buffer, because many times, due to alignment issues the allocated buffer a bit bigger than the requested size. Thus, the programmer can maximize the memory use using `allocation_command`. Here is the declaration of the function: [c++] enum boost::interprocess::allocation_type { //Bitwise OR (|) combinable values boost::interprocess::allocate_new = ..., boost::interprocess::expand_fwd = ..., boost::interprocess::expand_bwd = ..., boost::interprocess::shrink_in_place = ..., boost::interprocess::nothrow_allocation = ... }; template std::pair allocation_command( boost::interprocess::allocation_type command , std::size_t limit_size , size_type &prefer_in_recvd_out_size , T *&reuse_ptr); [*Preconditions for the function]: * If the parameter command contains the value `boost::interprocess::shrink_in_place` it can't contain any of these values: `boost::interprocess::expand_fwd`, `boost::interprocess::expand_bwd`. * If the parameter command contains `boost::interprocess::expand_fwd` or `boost::interprocess::expand_bwd`, the parameter `reuse_ptr` must be non-null and returned by a previous allocation function. * If the parameter command contains the value `boost::interprocess::shrink_in_place`, the parameter `limit_size` must be equal or greater than the parameter `preferred_size`. * If the parameter `command` contains any of these values: `boost::interprocess::expand_fwd` or `boost::interprocess::expand_bwd`, the parameter `limit_size` must be equal or less than the parameter `preferred_size`. [*Which are the effects of this function:] * If the parameter command contains the value `boost::interprocess::shrink_in_place`, the function will try to reduce the size of the memory block referenced by pointer `reuse_ptr` to the value `preferred_size` moving only the end of the block. If it's not possible, it will try to reduce the size of the memory block as much as possible as long as this results in `size(p) <= limit_size`. Success is reported only if this results in `preferred_size <= size(p)` and `size(p) <= limit_size`. * If the parameter `command` only contains the value `boost::interprocess::expand_fwd` (with optional additional `boost::interprocess::nothrow_allocation`), the allocator will try to increase the size of the memory block referenced by pointer reuse moving only the end of the block to the value `preferred_size`. If it's not possible, it will try to increase the size of the memory block as much as possible as long as this results in `size(p) >= limit_size`. Success is reported only if this results in `limit_size <= size(p)`. * If the parameter `command` only contains the value `boost::interprocess::expand_bwd` (with optional additional `boost::interprocess::nothrow_allocation`), the allocator will try to increase the size of the memory block referenced by pointer `reuse_ptr` only moving the start of the block to a returned new position `new_ptr`. If it's not possible, it will try to move the start of the block as much as possible as long as this results in `size(new_ptr) >= limit_size`. Success is reported only if this results in `limit_size <= size(new_ptr)`. * If the parameter `command` only contains the value `boost::interprocess::allocate_new` (with optional additional `boost::interprocess::nothrow_allocation`), the allocator will try to allocate memory for `preferred_size` objects. If it's not possible it will try to allocate memory for at least `limit_size` objects. * If the parameter `command` only contains a combination of `boost::interprocess::expand_fwd` and `boost::interprocess::allocate_new`, (with optional additional `boost::interprocess::nothrow_allocation`) the allocator will try first the forward expansion. If this fails, it would try a new allocation. * If the parameter `command` only contains a combination of `boost::interprocess::expand_bwd` and `boost::interprocess::allocate_new` (with optional additional `boost::interprocess::nothrow_allocation`), the allocator will try first to obtain `preferred_size` objects using both methods if necessary. If this fails, it will try to obtain `limit_size` objects using both methods if necessary. * If the parameter `command` only contains a combination of `boost::interprocess::expand_fwd` and `boost::interprocess::expand_bwd` (with optional additional `boost::interprocess::nothrow_allocation`), the allocator will try first forward expansion. If this fails it will try to obtain preferred_size objects using backwards expansion or a combination of forward and backwards expansion. If this fails, it will try to obtain `limit_size` objects using both methods if necessary. * If the parameter `command` only contains a combination of allocation_new, `boost::interprocess::expand_fwd` and `boost::interprocess::expand_bwd`, (with optional additional `boost::interprocess::nothrow_allocation`) the allocator will try first forward expansion. If this fails it will try to obtain preferred_size objects using new allocation, backwards expansion or a combination of forward and backwards expansion. If this fails, it will try to obtain `limit_size` objects using the same methods. * The allocator always writes the size or the expanded/allocated/shrunk memory block in `received_size`. On failure the allocator writes in `received_size` a possibly successful `limit_size` parameter for a new call. [*Throws an exception if two conditions are met:] * The allocator is unable to allocate/expand/shrink the memory or there is an error in preconditions * The parameter command does not contain `boost::interprocess::nothrow_allocation`. [*This function returns:] * The address of the allocated memory or the new address of the expanded memory as the first member of the pair. If the parameter command contains `boost::interprocess::nothrow_allocation` the first member will be 0 if the allocation/expansion fails or there is an error in preconditions. * The second member of the pair will be false if the memory has been allocated, true if the memory has been expanded. If the first member is 0, the second member has an undefined value. [*Notes:] * If the user chooses `char` as template argument the returned buffer will be suitably aligned to hold any type. * If the user chooses `char` as template argument and a backwards expansion is performed, although properly aligned, the returned buffer might not be suitable because the distance between the new beginning and the old beginning might not multiple of the type the user wants to construct, since due to internal restrictions the expansion can be slightly bigger than the requested bytes. [*When performing backwards expansion, if you have already constructed objects in the old buffer, make sure to specify correctly the type.] Here is a small example that shows the use of `allocation_command`: [import ../example/doc_managed_allocation_command.cpp] [doc_managed_allocation_command] `allocation_command` is a very powerful function that can lead to important performance gains. It's specially useful when programming vector-like data structures where the programmer can minimize both the number of allocation requests and the memory waste. [endsect] [section:copy_on_write_read_only Opening managed shared memory and mapped files with Copy On Write or Read Only modes] When mapping a memory segment based on shared memory or files, there is an option to open them using [*open_copy_on_write] option. This option is similar to `open_only` but every change the programmer does with this managed segment is kept private to this process and is not translated to the underlying device (shared memory or file). The underlying shared memory or file is opened as read-only so several processes can share an initial managed segment and make private changes to it. If many processes open a managed segment in copy on write mode and not modified pages from the managed segment will be shared between all those processes, with considerable memory savings. Opening managed shared memory and mapped files with [*open_read_only] maps the underlying device in memory with [*read-only] attributes. This means that any attempt to write that memory, either creating objects or locking any mutex might result in an page-fault error (and thus, program termination) from the OS. Read-only mode opens the underlying device (shared memory, file...) in read-only mode and can result in considerable memory savings if several processes just want to process a managed memory segment without modifying it. Read-only mode operations are limited: * Read-only mode must be used only from managed classes. If the programmer obtains the segment manager and tries to use it directly it might result in an access violation. The reason for this is that the segment manager is placed in the underlying device and does not nothing about the mode it's been mapped in memory. * Only const member functions from managed segments should be used. * Additionally, the `find<>` member function avoids using internal locks and can be used to look for named and unique objects. Here is an example that shows the use of these two open modes: [import ../example/doc_managed_copy_on_write.cpp] [doc_managed_copy_on_write] [endsect] [endsect] [section:managed_heap_memory_external_buffer Managed Heap Memory And Managed External Buffer] [*Boost.Interprocess] offers managed shared memory between processes using `managed_shared_memory` or `managed_mapped_file`. Two processes just map the same the memory mappable resource and read from and write to that object. Many times, we don't want to use that shared memory approach and we prefer to send serialized data through network, local socket or message queues. Serialization can be done through [*Boost.Serialization] or similar library. However, if two processes share the same ABI (application binary interface), we could use the same object and container construction capabilities of `managed_shared_memory` or `managed_heap_memory` to build all the information in a single buffer that will be sent, for example, though message queues. The receiver would just copy the data to a local buffer, and it could read or modify it directly without deserializing the data . This approach can be much more efficient that a complex serialization mechanism. Applications for [*Boost.Interprocess] services using non-shared memory buffers: * Create and use STL compatible containers and allocators, in systems where dynamic memory is not recommendable. * Build complex, easily serializable databases in a single buffer: * To share data between threads * To save and load information from/to files. * Duplicate information (containers, allocators, etc...) just copying the contents of one buffer to another one. * Send complex information and objects/databases using serial/inter-process/network communications. To help with this management, [*Boost.Interprocess] provides two useful classes, `basic_managed_heap_memory` and `basic_managed_external_buffer`: [section:managed_external_buffer Managed External Buffer: Constructing all Boost.Interprocess objects in a user provided buffer] Sometimes, the user wants to create simple objects, STL compatible containers, STL compatible strings and more, all in a single buffer. This buffer could be a big static buffer, a memory-mapped auxiliary device or any other user buffer. This would allow an easy serialization and we-ll just need to copy the buffer to duplicate all the objects created in the original buffer, including complex objects like maps, lists.... [*Boost.Interprocess] offers managed memory segment classes to handle user provided buffers that allow the same functionality as shared memory classes: [c++] //Named object creation managed memory segment //All objects are constructed in a user provided buffer template < class CharType, class MemoryAlgorithm, template class IndexType > class basic_managed_external_buffer; //Named object creation managed memory segment //All objects are constructed in a user provided buffer // Names are c-strings, // Default memory management algorithm // (rbtree_best_fit with no mutexes and relative pointers) // Name-object mappings are stored in the default index type (flat_map) typedef basic_managed_external_buffer < char, rbtree_best_fit >, flat_map_index > managed_external_buffer; //Named object creation managed memory segment //All objects are constructed in a user provided buffer // Names are wide-strings, // Default memory management algorithm // (rbtree_best_fit with no mutexes and relative pointers) // Name-object mappings are stored in the default index type (flat_map) typedef basic_managed_external_buffer< wchar_t, rbtree_best_fit >, flat_map_index > wmanaged_external_buffer; To use a managed external buffer, you must include the following header: [c++] #include Let's see an example of the use of managed_external_buffer: [import ../example/doc_managed_external_buffer.cpp] [doc_managed_external_buffer] [*Boost.Interprocess] STL compatible allocators can also be used to place STL compatible containers in the user segment. [classref boost::interprocess::basic_managed_external_buffer basic_managed_external_buffer] can be also useful to build small databases for embedded systems limiting the size of the used memory to a predefined memory chunk, instead of letting the database fragment the heap memory. [endsect] [section:managed_heap_memory Managed Heap Memory: Boost.Interprocess machinery in heap memory] The use of heap memory (new/delete) to obtain a buffer where the user wants to store all his data is very common, so [*Boost.Interprocess] provides some specialized classes that work exclusively with heap memory. These are the classes: [c++] //Named object creation managed memory segment //All objects are constructed in a single buffer allocated via new[] template < class CharType, class MemoryAlgorithm, template class IndexType > class basic_managed_heap_memory; //Named object creation managed memory segment //All objects are constructed in a single buffer allocated via new[] // Names are c-strings, // Default memory management algorithm // (rbtree_best_fit with no mutexes and relative pointers) // Name-object mappings are stored in the default index type (flat_map) typedef basic_managed_heap_memory < char, rbtree_best_fit, flat_map_index > managed_heap_memory; //Named object creation managed memory segment //All objects are constructed in a single buffer allocated via new[] // Names are wide-strings, // Default memory management algorithm // (rbtree_best_fit with no mutexes and relative pointers) // Name-object mappings are stored in the default index type (flat_map) typedef basic_managed_heap_memory< wchar_t, rbtree_best_fit, flat_map_index > wmanaged_heap_memory; To use a managed heap memory, you must include the following header: [c++] #include The use is exactly the same as [classref boost::interprocess::basic_managed_external_buffer basic_managed_external_buffer], except that memory is created by the managed memory segment itself using dynamic (new/delete) memory. [*basic_managed_heap_memory] also offers a `grow(std::size_t extra_bytes)` function that tries to resize internal heap memory so that we have room for more objects. But *be careful*, if memory is reallocated, the old buffer will be copied into the new one so all the objects will be binary-copied to the new buffer. To be able to use this function, all pointers constructed in the heap buffer that point to objects in the heap buffer must be relative pointers (for example `offset_ptr`). Otherwise, the result is undefined. Here is an example: [import ../example/doc_managed_heap_memory.cpp] [doc_managed_heap_memory] [endsect] [section:managed_heap_memory_external_buffer_diff Differences between managed memory segments] All managed memory segments have similar capabilities (memory allocation inside the memory segment, named object construction...), but there are some remarkable differences between [*managed_shared_memory], [*managed_mapped_file] and [*managed_heap_memory], [*managed_external_file]. * Default specializations of managed shared memory and mapped file use process-shared mutexes. Heap memory and external buffer have no internal synchronization by default. The cause is that the first two are thought to be shared between processes (although memory mapped files could be used just to obtain a persistent object data-base for a process) whereas the last two are thought to be used inside one process to construct a serialized named object data-base that can be sent though serial interprocess communications (like message queues, localhost network...). * The first two create a system-global object (a shared memory object or a file) shared by several processes, whereas the last two are objects that don't create system-wide resources. [endsect] [section:shared_message_queue_ex Example: Serializing a database through the message queue] To see the utility of managed heap memory and managed external buffer classes, the following example shows how a message queue can be used to serialize a whole database constructed in a memory buffer using [*Boost.Interprocess], send the database through a message queue and duplicated in another buffer: [import ../test/message_queue_test.cpp] [message_queue_test_test_serialize_db] [endsect] [endsect] [endsect] [section:allocators_containers Allocators, containers and memory allocation algorithms] [section:allocator_introduction Introduction to Interprocess allocators] As seen, [*Boost.Interprocess] offers raw memory allocation and object construction using managed memory segments (managed shared memory, managed mapped files...) and one of the first user requests is the use of containers in managed shared memories. To achieve this, [*Boost.Interprocess] makes use of managed memory segment's memory allocation algorithms to build several memory allocation schemes, including general purpose and node allocators. [*Boost.Interprocess] STL compatible allocators are configurable via template parameters. Allocators define their `pointer` typedef based on the `void_pointer` typedef of the segment manager passed as template argument. When this `segment_manager::void_pointer` is a relative pointer, (for example, `offset_ptr`) the user can place these allocators in memory mapped in different base addresses in several processes. [section:allocator_properties Properties of [*Boost.Interprocess] allocators] Container allocators are normally default-constructible because the are stateless. `std::allocator` and [*Boost.Pool's] `boost::pool_allocator`/`boost::fast_pool_allocator` are examples of default-constructible allocators. On the other hand, [*Boost.Interprocess] allocators need to allocate memory from a concrete memory segment and not from a system-wide memory source (like the heap). [*Boost.Interprocess] allocators are [*stateful], which means that they must be configured to tell them where the shared memory or the memory mapped file is. This information is transmitted at compile-time and run-time: The allocators receive a template parameter defining the type of the segment manager and their constructor receive a pointer to the segment manager of the managed memory segment where the user wants to allocate the values. [*Boost.Interprocess] allocators have [*no default-constructors] and containers must be explicitly initialized with a configured allocator: [c++] //The allocators must be templatized with the segment manager type typedef any_interprocess_allocator Allocator; //The allocator must be constructed with a pointer to the segment manager Allocator alloc_instance (segment.get_segment_manager(), ...); //Containers must be initialized with a configured allocator typedef my_list MyIntList; MyIntList mylist(alloc_inst); //This would lead to a compilation error, because //the allocator has no default constructor //MyIntList mylist; [*Boost.Interprocess] allocators also have a `get_segment_manager()` function that returns the underlying segment manager that they have received in the constructor: [c++] Allocator::segment_manager s = alloc_instance.get_segment_manager(); AnotherType *a = s->construct(anonymous_instance)(/*Parameters*/); [endsect] [section:allocator_swapping Swapping Boost.Interprocess allocators] When swapping STL containers, there is an active discussion on what to do with the allocators. Some STL implementations, for example Dinkumware from Visual .NET 2003, perform a deep swap of the whole container through a temporary when allocators are not equal. The [@http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2004/n1599.html proposed resolution] to container swapping is that allocators should be swapped in a non-throwing way. Unfortunately, this approach is not valid with shared memory. Using heap allocators, if Group1 of node allocators share a common segregated storage, and Group2 share another common segregated storage, a simple pointer swapping is needed to swap an allocator of Group1 and another allocator of Group2. But when the user wants to swap two shared memory allocators, each one placed in a different shared memory segment, this is not possible. As generally shared memory is mapped in different addresses in each process, a pointer placed in one segment can't point to any object placed in other shared memory segment, since in each process, the distance between the segments is different. However, if both shared memory allocators are in the same segment, a non-throwing swap is possible, just like heap allocators. Until a final resolution is achieved. [*Boost.Interprocess] allocators implement a non-throwing swap function that swaps internal pointers. If an allocator placed in a shared memory segment is swapped with other placed in a different shared memory segment, the result is undefined. But a crash is quite sure. [endsect] [section:allocator allocator: A general purpose allocator for managed memory segments] The [classref boost::interprocess::allocator allocator] class defines an allocator class that uses the managed memory segment's algorithm to allocate and deallocate memory. This is achieved through the [*segment manager] of the managed memory segment. This allocator is the equivalent for managed memory segments of the standard `std::allocator`. [classref boost::interprocess::allocator allocator] is templatized with the allocated type, and the segment manager. [*Equality:] Two [classref boost::interprocess::allocator allocator] instances constructed with the same segment manager compare equal. If an instance is created using copy constructor, that instance compares equal with the original one. [*Allocation thread-safety:] Allocation and deallocation are implemented as calls to the segment manager's allocation function so the allocator offers the same thread-safety as the segment manager. To use [classref boost::interprocess::allocator allocator] you must include the following header: [c++] #include [classref boost::interprocess::allocator allocator] has the following declaration: [c++] namespace boost { namespace interprocess { template class allocator; } //namespace interprocess { } //namespace boost { The allocator just provides the needed typedefs and forwards all allocation and deallocation requests to the segment manager passed in the constructor, just like `std::allocator` forwards the requests to `operator new[]`. Using [classref boost::interprocess::allocator allocator] is straightforward: [import ../example/doc_allocator.cpp] [doc_allocator] [endsect] [endsect] [section:stl_allocators_segregated_storage Segregated storage node allocators] Variable size memory algorithms waste some space in management information for each allocation. Sometimes, usually for small objects, this is not acceptable. Memory algorithms can also fragment the managed memory segment under some allocation and deallocation schemes, reducing their performance. When allocating many objects of the same type, a simple segregated storage becomes a fast and space-friendly allocator, as explained in the [@http://www.boost.org/libs/pool/ [*Boost.Pool]] library. Segregate storage node allocators allocate large memory chunks from a general purpose memory allocator and divide that chunk into several nodes. No bookkeeping information is stored in the nodes to achieve minimal memory waste: free nodes are linked using a pointer constructed in the memory of the node. [*Boost.Interprocess] offers 3 allocators based on this segregated storage algorithm: [classref boost::interprocess::node_allocator node_allocator], [classref boost::interprocess::private_node_allocator private_node_allocator] and [classref boost::interprocess::cached_node_allocator cached_node_allocator]. To know the details of the implementation of of the segregated storage pools see the [link interprocess.architecture.allocators_containers.implementation_segregated_storage_pools Implementation of [*Boost.Interprocess] segregated storage pools] section. [section:segregated_allocators_common Additional parameters and functions of segregated storage node allocators] [classref boost::interprocess::node_allocator node_allocator], [classref boost::interprocess::private_node_allocator private_node_allocator] and [classref boost::interprocess::cached_node_allocator cached_node_allocator] implement the standard allocator interface and the functions explained in the [link interprocess.allocators_containers.allocator_introduction.allocator_properties Properties of Boost.Interprocess allocators]. All these allocators are templatized by 3 parameters: * `class T`: The type to be allocated. * `class SegmentManager`: The type of the segment manager that will be passed in the constructor. * `std::size_t NodesPerChunk`: The number of nodes that a memory chunk will contain. This value will define the size of the memory the pool will request to the segment manager when the pool runs out of nodes. This parameter has a default value. These allocators also offer the `deallocate_free_chunks()` function. This function will traverse all the memory chunks of the pool and will return to the managed memory segment the free chunks of memory. If this function is not used, deallocating the free chunks does not happen until the pool is destroyed so the only way to return memory allocated by the pool to the segment before destructing the pool is calling manually this function. This function is quite time-consuming because it has quadratic complexity (O(N^2)). [endsect] [section:node_allocator node_allocator: A process-shared segregated storage] For heap-memory node allocators (like [*Boost.Pool's] `boost::fast_pool_allocator` usually a global, thread-shared singleton pool is used for each node size. This is not possible if you try to share a node allocator between processes. To achieve this sharing [classref boost::interprocess::node_allocator node_allocator] uses the segment manager's unique type allocation service (see [link interprocess.managed_memory_segments.managed_memory_segment_features.unique Unique instance construction] section). In the initialization, a [classref boost::interprocess::node_allocator node_allocator] object searches this unique object in the segment. If it is not preset, it builds one. This way, all [classref boost::interprocess::node_allocator node_allocator] objects built inside a memory segment share a unique memory pool. The common segregated storage is not only shared between node_allocators of the same type, but it is also shared between all node allocators that allocate objects of the same size, for example, [*node_allocator] and [*node_allocator]. This saves a lot of memory but also imposes an synchronization overhead for each node allocation. The dynamically created common segregated storage integrates a reference count so that a [classref boost::interprocess::node_allocator node_allocator] can know if any other [classref boost::interprocess::node_allocator node_allocator] is attached to the same common segregated storage. When the last allocator attached to the pool is destroyed, the pool is destroyed. [*Equality:] Two [classref boost::interprocess::node_allocator node_allocator] instances constructed with the same segment manager compare equal. If an instance is created using copy constructor, that instance compares equal with the original one. [*Allocation thread-safety:] Allocation and deallocation are implemented as calls to the shared pool. The shared pool offers the same synchronization guarantees as the segment manager. To use [classref boost::interprocess::node_allocator node_allocator], you must include the following header: [c++] #include [classref boost::interprocess::node_allocator node_allocator] has the following declaration: [c++] namespace boost { namespace interprocess { template class node_allocator; } //namespace interprocess { } //namespace boost { An example using [classref boost::interprocess::node_allocator node_allocator]: [import ../example/doc_node_allocator.cpp] [doc_node_allocator] [endsect] [section:private_node_allocator private_node_allocator: a private segregated storage] As said, the node_allocator shares a common segregated storage between node_allocators that allocate objects of the same size and this optimizes memory usage. However, it needs a unique/named object construction feature so that this sharing can be possible. Also imposes a synchronization overhead per node allocation because of this share. Sometimes, the unique object service is not available (for example, when building index types to implement the named allocation service itself) or the synchronization overhead is not acceptable. Many times the programmer wants to make sure that the pool is destroyed when the allocator is destroyed, to free the memory as soon as possible. So [*private_node_allocator] uses the same segregated storage as `node_allocator`, but each [*private_node_allocator] has its own segregated storage pool. No synchronization is used when allocating nodes, so there is far less overhead for an operation that usually involves just a few pointer operations when allocating and deallocating a node. [*Equality:] Two [classref boost::interprocess::private_node_allocator private_node_allocator] instances [*never] compare equal. Memory allocated with one allocator [*can't] be deallocated with another one. [*Allocation thread-safety:] Allocation and deallocation are [*not] thread-safe. To use [classref boost::interprocess::private_node_allocator private_node_allocator], you must include the following header: [c++] #include [classref boost::interprocess::private_node_allocator private_node_allocator] has the following declaration: [c++] namespace boost { namespace interprocess { template class private_node_allocator; } //namespace interprocess { } //namespace boost { An example using [classref boost::interprocess::private_node_allocator private_node_allocator]: [import ../example/doc_private_node_allocator.cpp] [doc_private_node_allocator] [endsect] [section:cached_node_allocator cached_node_allocator: caching nodes to avoid overhead] The total node sharing of [classref boost::interprocess::node_allocator node_allocator] can impose a high overhead for some applications and the minimal synchronization overhead of [classref boost::interprocess::private_node_allocator private_node_allocator] can impose a unacceptable memory waste for other applications. To solve this, [*Boost.Interprocess] offers an allocator, [classref boost::interprocess::cached_node_allocator cached_node_allocator], that allocates nodes from the common pool but caches some of them privately so that following allocations have no synchronization overhead. When the cache is full, the allocator returns some cached nodes to the common pool, and those will be available to other allocators. [*Equality:] Two [classref boost::interprocess::cached_node_allocator cached_node_allocator] instances constructed with the same segment manager compare equal. If an instance is created using copy constructor, that instance compares equal with the original one. [*Allocation thread-safety:] Allocation and deallocation are [*not] thread-safe. To use [classref boost::interprocess::cached_node_allocator cached_node_allocator], you must include the following header: [c++] #include [classref boost::interprocess::cached_node_allocator cached_node_allocator] has the following declaration: [c++] namespace boost { namespace interprocess { template class cached_node_allocator; } //namespace interprocess { } //namespace boost { A [classref boost::interprocess::cached_node_allocator cached_node_allocator] instance and a [classref boost::interprocess::node_allocator node_allocator] instance share the same pool if both instances receive the same template parameters. This means that nodes returned to the shared pool by one of them can be reused by the other. Please note that this does not mean that both allocators compare equal, this is just information for programmers that want to maximize the use of the pool. [classref boost::interprocess::cached_node_allocator cached_node_allocator], offers additional functions to control the cache (the cache can be controlled per instance): * `void set_max_cached_nodes(std::size_t n)`: Sets the maximum cached nodes limit. If cached nodes reach the limit, some are returned to the shared pool. * `std::size_t get_max_cached_nodes() const`: Returns the maximum cached nodes limit. * `void deallocate_cache()`: Returns the cached nodes to the shared pool. An example using [classref boost::interprocess::cached_node_allocator cached_node_allocator]: [import ../example/doc_cached_node_allocator.cpp] [doc_cached_node_allocator] [endsect] [endsect] [section:stl_allocators_adaptive Adaptive pool node allocators] Node allocators based on simple segregated storage algorithm are both space-efficient and fast but they have a problem: they only can grow. Every allocated node avoids any payload to store additional data and that leads to the following limitation: when a node is deallocated, it's stored in a free list of nodes but memory is not returned to the segment manager so a deallocated node can be only reused by other containers using the same node pool. This behaviour can be problematic if several containers use [classref boost::interprocess::node_allocator] to temporarily allocate a lot of objects but they end storing a few of them: the node pool will be full of nodes that won't be reused wasting memory from the segment. Adaptive pool based allocators trade some space (the overhead can be as low as 1%) and performance (acceptable for many applications) with the ability to return free chunks of nodes to the memory segment, so that they can be used by any other container or managed object construction. To know the details of the implementation of of "adaptive pools" see the [link interprocess.architecture.allocators_containers.implementation_adaptive_pools Implementation of [*Boost.Intrusive] adaptive pools] section. Like with segregated storage based node allocators, Boost.Interprocess offers 3 new allocators: [classref boost::interprocess::adaptive_pool adaptive_pool], [classref boost::interprocess::private_adaptive_pool private_adaptive_pool], [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool]. [section:adaptive_allocators_common Additional parameters and functions of adaptive pool node allocators] [classref boost::interprocess::adaptive_pool adaptive_pool], [classref boost::interprocess::private_adaptive_pool private_adaptive_pool] and [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool] implement the standard allocator interface and the functions explained in the [link interprocess.allocators_containers.allocator_introduction.allocator_properties Properties of Boost.Interprocess allocators]. All these allocators are templatized by 4 parameters: * `class T`: The type to be allocated. * `class SegmentManager`: The type of the segment manager that will be passed in the constructor. * `std::size_t NodesPerChunk`: The number of nodes that a memory chunk will contain. This value will define the size of the memory the pool will request to the segment manager when the pool runs out of nodes. This parameter has a default value. * `std::size_t MaxFreeChunks`: The maximum number of free chunks that the pool will hold. If this limit is reached the pool returns the chunks to the segment manager. This parameter has a default value. These allocators also offer the `deallocate_free_chunks()` function. This function will traverse all the memory chunks of the pool and will return to the managed memory segment the free chunks of memory. This function is much faster than for segregated storage allocators, because the adaptive pool algorithm offers constant-time access to free chunks. [endsect] [section:adaptive_pool adaptive_pool: a process-shared adaptive pool] Just like [classref boost::interprocess::node_allocator node_allocator] a global, process-thread pool is used for each node size. In the initialization, [classref boost::interprocess::adaptive_pool adaptive_pool] searches the pool in the segment. If it is not preset, it builds one. The adaptive pool, is created using a unique name. The adaptive pool it is also shared between all node_allocators that allocate objects of the same size, for example, [*adaptive_pool] and [*adaptive_pool]. The common adaptive pool is destroyed when all the allocators attached to the pool are destroyed. [*Equality:] Two [classref boost::interprocess::adaptive_pool adaptive_pool] instances constructed with the same segment manager compare equal. If an instance is created using copy constructor, that instance compares equal with the original one. [*Allocation thread-safety:] Allocation and deallocation are implemented as calls to the shared pool. The shared pool offers the same synchronization guarantees as the segment manager. To use [classref boost::interprocess::adaptive_pool adaptive_pool], you must include the following header: [c++] #include [classref boost::interprocess::adaptive_pool adaptive_pool] has the following declaration: [c++] namespace boost { namespace interprocess { template class adaptive_pool; } //namespace interprocess { } //namespace boost { An example using [classref boost::interprocess::adaptive_pool adaptive_pool]: [import ../example/doc_adaptive_pool.cpp] [doc_adaptive_pool] [endsect] [section:private_adaptive_pool private_adaptive_pool: a private adaptive pool] Just like [classref boost::interprocess::private_node_allocator private_node_allocator] owns a private segregated storage pool, [classref boost::interprocess::private_adaptive_pool private_adaptive_pool] owns its own adaptive pool. If the user wants to avoid the excessive node allocation synchronization overhead in a container [classref boost::interprocess::private_adaptive_pool private_adaptive_pool] is a good choice. [*Equality:] Two [classref boost::interprocess::private_adaptive_pool private_adaptive_pool] instances [*never] compare equal. Memory allocated with one allocator [*can't] be deallocated with another one. [*Allocation thread-safety:] Allocation and deallocation are [*not] thread-safe. To use [classref boost::interprocess::private_adaptive_pool private_adaptive_pool], you must include the following header: [c++] #include [classref boost::interprocess::private_adaptive_pool private_adaptive_pool] has the following declaration: [c++] namespace boost { namespace interprocess { template class private_adaptive_pool; } //namespace interprocess { } //namespace boost { An example using [classref boost::interprocess::private_adaptive_pool private_adaptive_pool]: [import ../example/doc_private_adaptive_pool.cpp] [doc_private_adaptive_pool] [endsect] [section:cached_adaptive_pool cached_adaptive_pool: Avoiding synchronization overhead] Adaptive pools have also a cached version. In this allocator the allocator caches some nodes to avoid the synchronization and bookkeeping overhead of the shared adaptive pool. [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool] allocates nodes from the common adaptive pool but caches some of them privately so that following allocations have no synchronization overhead. When the cache is full, the allocator returns some cached nodes to the common pool, and those will be available to other [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pools] or [classref boost::interprocess::adaptive_pool adaptive_pools] of the same managed segment. [*Equality:] Two [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool] instances constructed with the same segment manager compare equal. If an instance is created using copy constructor, that instance compares equal with the original one. [*Allocation thread-safety:] Allocation and deallocation are [*not] thread-safe. To use [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool], you must include the following header: [c++] #include [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool] has the following declaration: [c++] namespace boost { namespace interprocess { template class cached_adaptive_pool; } //namespace interprocess { } //namespace boost { A [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool] instance and an [classref boost::interprocess::adaptive_pool adaptive_pool] instance share the same pool if both instances receive the same template parameters. This means that nodes returned to the shared pool by one of them can be reused by the other. Please note that this does not mean that both allocators compare equal, this is just information for programmers that want to maximize the use of the pool. [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool], offers additional functions to control the cache (the cache can be controlled per instance): * `void set_max_cached_nodes(std::size_t n)`: Sets the maximum cached nodes limit. If cached nodes reach the limit, some are returned to the shared pool. * `std::size_t get_max_cached_nodes() const`: Returns the maximum cached nodes limit. * `void deallocate_cache()`: Returns the cached nodes to the shared pool. An example using [classref boost::interprocess::cached_adaptive_pool cached_adaptive_pool]: [import ../example/doc_cached_adaptive_pool.cpp] [doc_cached_adaptive_pool] [endsect] [endsect] [section:containers_explained Interprocess and containers in managed memory segments] [section:stl_container_requirements Container requirements for Boost.Interprocess allocators] [*Boost.Interprocess] STL compatible allocators offer a STL compatible allocator interface and if they define their internal *pointer* typedef as a relative pointer, they can be used to place STL containers in shared memory, memory mapped files or in a user defined memory segment. However, as Scott Meyers mentions in his Effective STL book, Item 10, ['"Be aware of allocator conventions and restrictions"]: * ['"the Standard explicitly allows library implementers to assume that every allocator's pointer typedef is a synonym for T*"] * ['"the Standard says that an implementation of the STL is permitted to assume that all allocator objects of the same type are equivalent and always compare equal"] Obviously, if any STL implementation ignores pointer typedefs, no smart pointer can be used as allocator::pointer. If STL implementations assume all allocator objects of the same type compare equal, it will assume that two allocators, each one allocating from a different memory pool are equal, which is a complete disaster. STL containers that we want to place in shared memory or memory mapped files with [*Boost.Interprocess] can't make any of these assumptions, so: * STL containers may not assume that memory allocated with an allocator can be deallocated with other allocators of the same type. All allocators objects must compare equal only if memory allocated with one object can be deallocated with the other one, and this can only tested with operator==() at run-time. * Containers' internal pointers should be of the type allocator::pointer and containers may not assume allocator::pointer is a raw pointer. * All objects must be constructed-destroyed via allocator::construct and allocator::destroy functions. [endsect] [section:containers STL containers in managed memory segments] Unfortunately, many STL implementations use raw pointers for internal data and ignore allocator pointer typedefs and others suppose at some point that the allocator::typedef is T*. This is because in practice, there wasn't need of allocators with a pointer typedef different from T* for pooled/node memory allocators. Until STL implementations handle allocator::pointer typedefs in a generic way, [*Boost.Interprocess] offers the following classes: * [*boost:interprocess::vector] is the implementation of `std::vector` ready to be used in managed memory segments like shared memory. To use it include: [c++] #include * [*boost:interprocess::deque] is the implementation of `std::deque` ready to be used in managed memory segments like shared memory. To use it include: [c++] #include * [classref boost::interprocess::list list] is the implementation of `std::list` ready to be used in managed memory segments like shared memory. To use it include: [c++] #include * [classref boost::interprocess::slist slist] is the implementation of SGI's `slist` container (singly linked list) ready to be used in managed memory segments like shared memory. To use it include: [c++] #include * [classref boost::interprocess::set set]/ [classref boost::interprocess::multiset multiset]/ [classref boost::interprocess::map map]/ [classref boost::interprocess::multimap multimap] family is the implementation of std::set/multiset/map/multimap family ready to be used in managed memory segments like shared memory. To use them include: [c++] #include #include * [classref boost::interprocess::flat_set flat_set]/ [classref boost::interprocess::flat_multiset flat_multiset]/ [classref boost::interprocess::flat_map flat_map]/ [classref boost::interprocess::flat_multimap flat_multimap] classes are the adaptation and extension of Andrei Alexandrescu's famous AssocVector class from Loki library, ready for the shared memory. These classes offer the same functionality as `std::set/multiset/map/multimap` implemented with an ordered vector, which has faster lookups than the standard ordered associative containers based on red-black trees, but slower insertions. To use it include: [c++] #include #include * [classref boost::interprocess::basic_string basic_string] is the implementation of `std::basic_string` ready to be used in managed memory segments like shared memory. It's implemented using a vector-like contiguous storage, so it has fast c string conversion and can be used with the [link interprocess.streams.vectorstream vectorstream] iostream formatting classes. To use it include: [c++] #include All these containers have the same default arguments as standard containers and they can be used with other, non [*Boost.Interprocess] allocators (std::allocator, or boost::pool_allocator, for example). To place any of these containers in managed memory segments, we must define the allocator template parameter with a [*Boost.Interprocess] allocator so that the container allocates the values in the managed memory segment. To place the container itself in shared memory, we construct it in the managed memory segment just like any other object with [*Boost.Interprocess]: [import ../example/doc_cont.cpp] [doc_cont] These containers also show how easy is to create/modify an existing container making possible to place it in shared memory. [endsect] [section:where_allocate Where is this being allocated?] [*Boost.Interprocess] containers are placed in shared memory/memory mapped files, etc... using two mechanisms [*at the same time]: * [*Boost.Interprocess ]`construct<>`, `find_or_construct<>`... functions. These functions place a C++ object in the shared memory/memory mapped file. But this places only the object, but *not* the memory that this object may allocate dynamically. * Shared memory allocators. These allow allocating shared memory/memory mapped file portions so that containers can allocate dynamically fragments of memory to store newly inserted elements. This means that to place any [*Boost.Interprocess] container (including [*Boost.Interprocess] strings) in shared memory or memory mapped files, containers *must*: * Define their template allocator parameter to a [*Boost.Interprocess] allocator. * Every container constructor must take the [*Boost.Interprocess] allocator as parameter. * You must use construct<>/find_or_construct<>... functions to place the container in the managed memory. If you do the first two points but you don't use `construct<>` or `find_or_construct<>` you are creating a container placed *only* in your process but that allocates memory for contained types from shared memory/memory mapped file. Let's see an example: [import ../example/doc_where_allocate.cpp] [doc_where_allocate] [endsect] [section:containers_and_move Move semantics in Interprocess containers] [*Boost.Interprocess] containers support move semantics, which means that the contents of a container can be moved from a container to another one, without any copying. The contents of the source container are transferred to the target container and the source container is left in default-constructed state. When using containers of containers, we can also use move-semantics to insert objects in the container, avoiding unnecessary copies. To transfer the contents of a container to another one, use `boost::move()` function, as shown in the example. For more details about functions supporting move-semantics, see the reference section of Boost.Interprocess containers: [import ../example/doc_move_containers.cpp] [doc_move_containers] [endsect] [section:containers_of_containers Containers of containers] When creating containers of containers, each container needs an allocator. To avoid using several allocators with complex type definitions, we can take advantage of the type erasure provided by void allocators and the ability to implicitly convert void allocators in allocators that allocate other types. Here we have an example that builds a map in shared memory. Key is a string and the mapped type is a class that stores several containers: [import ../example/doc_complex_map.cpp] [doc_complex_map] [endsect] [endsect] [section:additional_containers Boost containers compatible with Boost.Interprocess] As mentioned, container developers might need to change their implementation to make them compatible with Boost.Interprocess, because implementation usually ignore allocators with smart pointers. Hopefully several Boost containers are compatible with [*Interprocess]. [section:unordered Boost unordered containers] [*Boost.Unordered] containers are compatible with Interprocess, so programmers can store hash containers in shared memory and memory mapped files. Here is a small example storing `unordered_map` in shared memory: [import ../example/doc_unordered_map.cpp] [doc_unordered_map] [endsect] [section:multi_index Boost.MultiIndex containers] The widely used [*Boost.MultiIndex] library is compatible with [*Boost.Interprocess] so we can construct pretty good databases in shared memory. Constructing databases in shared memory is a bit tougher than in normal memory, usually because those databases contain strings and those strings need to be placed in shared memory. Shared memory strings require an allocator in their constructors so this usually makes object insertion a bit more complicated. Here is an example that shows how to put a multi index container in shared memory: [import ../example/doc_multi_index.cpp] [doc_multi_index] [endsect] Programmers can place [*Boost.CircularBuffer] containers in sharecd memory provided they disable debugging facilities with defines `BOOST_CB_DISABLE_DEBUG` or the more general `NDEBUG`. The reason is that those debugging facilities are only compatible with raw pointers. [endsect] [endsect] [section:memory_algorithms Memory allocation algorithms] [section:simple_seq_fit simple_seq_fit: A simple shared memory management algorithm] The algorithm is a variation of sequential fit using singly linked list of free memory buffers. The algorithm is based on the article about shared memory titled [@http://home.earthlink.net/~joshwalker1/writing/SharedMemory.html ['"Taming Shared Memory"] ]. The algorithm is as follows: The shared memory is divided in blocks of free shared memory, each one with some control data and several bytes of memory ready to be used. The control data contains a pointer (in our case offset_ptr) to the next free block and the size of the block. The allocator consists of a singly linked list of free blocks, ordered by address. The last block, points always to the first block: [c++] simple_seq_fit memory layout: main extra allocated free_block_1 allocated free_block_2 allocated free_block_3 header header block ctrl usr block ctrl usr block ctrl usr _________ _____ _________ _______________ _________ _______________ _________ _______________ | || || || | || || | || || | | |free|ctrl||extra|| ||next|size| mem || ||next|size| mem || ||next|size| mem | |_________||_____||_________||_________|_____||_________||_________|_____||_________||_________|_____| | | | | | | | |_>_>_>_>_>_>_>_>_>_>_>_>_| |_>_>_>_>_>_>_>_>_>_>_>_>_| |_>_>_>_>_>_>_>_>_>_>_>_| | | | |_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<_<__| When a user requests N bytes of memory, the allocator traverses the free block list looking for a block large enough. If the "mem" part of the block has the same size as the requested memory, we erase the block from the list and return a pointer to the "mem" part of the block. If the "mem" part size is bigger than needed, we split the block in two blocks, one of the requested size and the other with remaining size. Now, we take the block with the exact size, erase it from list and give it to the user. When the user deallocates a block, we traverse the list (remember that the list is ordered), and search its place depending on the block address. Once found, we try to merge the block with adjacent blocks if possible. To ease implementation, the size of the free memory block is measured in multiples of "basic_size" bytes. The basic size will be the size of the control block aligned to machine most restrictive alignment. This algorithm is a low size overhead algorithm suitable for simple allocation schemes. This algorithm should only be used when size is a major concern, because the performance of this algorithm suffers when the memory is fragmented. This algorithm has linear allocation and deallocation time, so when the number of allocations is high, the user should use a more performance-friendly algorithm. In most 32 systems, with 8 byte alignment, "basic_size" is 8 bytes. This means that an allocation request of 1 byte leads to the creation of a 16 byte block, where 8 bytes are available to the user. The allocation of 8 bytes leads also to the same 16 byte block. [endsect] [section:rbtree_best_fit rbtree_best_fit: Best-fit logarithmic-time complexity allocation] This algorithm is an advanced algorithm using red-black trees to sort the free portions of the memory segment by size. This allows logarithmic complexity allocation. Apart from this, a doubly-linked list of all portions of memory (free and allocated) is maintained to allow constant-time access to previous and next blocks when doing merging operations. The data used to create the red-black tree of free nodes is overwritten by the user since it's no longer used once the memory is allocated. This maintains the memory size overhead down to the doubly linked list overhead, which is pretty small (two pointers). Basically this is the scheme: [c++] rbtree_best_fit memory layout: main allocated block free block allocated block free block header _______________ _______________ _________________________________ _______________ _________________________________ | || | || | | || | || | | | | main header ||next|prev| mem ||next|prev|left|right|parent| mem ||next|prev| mem ||next|prev|left|right|parent| mem | |_______________||_________|_____||_________|_________________|_____||_________|_____||_________|_________________|_____| This allocation algorithm is pretty fast and scales well with big shared memory segments and big number of allocations. To form a block a minimum memory size is needed: the sum of the doubly linked list and the red-black tree control data. The size of a block is measured in multiples of the most restrictive alignment value. In most 32 systems with 8 byte alignment the minimum size of a block is 24 byte. When a block is allocated the control data related to the red black tree is overwritten by the user (because it's only needed for free blocks). In those systems a 1 byte allocation request means that: * 24 bytes of memory from the segment are used to form a block. * 16 bytes of them are usable for the user. For really small allocations (<= 8 bytes), this algorithm wastes more memory than the simple sequential fit algorithm (8 bytes more). For allocations bigger than 8 bytes the memory overhead is exactly the same. This is the default allocation algorithm in [*Boost.Interprocess] managed memory segments. [endsect] [endsect] [section:streams Direct iostream formatting: vectorstream and bufferstream] Shared memory, memory-mapped files and all [*Boost.Interprocess] mechanisms are focused on efficiency. The reason why shared memory is used is that it's the fastest IPC mechanism available. When passing text-oriented messages through shared memory, there is need to format the message. Obviously C++ offers the iostream framework for that work. Some programmers appreciate the iostream safety and design for memory formatting but feel that the stringstream family is far from efficient not when formatting, but when obtaining formatted data to a string, or when setting the string from which the stream will extract data. An example: [c++] //Some formatting elements std::string my_text = "..."; int number; //Data reader std::istringstream input_processor; //This makes a copy of the string. If not using a //reference counted string, this is a serious overhead. input_processor.str(my_text); //Extract data while(/*...*/){ input_processor >> number; } //Data writer std::ostringstream output_processor; //Write data while(/*...*/){ output_processor << number; } //This returns a temporary string. Even with return-value //optimization this is expensive. my_text = input_processor.str(); The problem is even worse if the string is a shared-memory string, because to extract data, we must copy the data first from shared-memory to a `std::string` and then to a `std::stringstream`. To encode data in a shared memory string we should copy data from a `std::stringstream` to a `std::string` and then to the shared-memory string. Because of this overhead, [*Boost.Interprocess] offers a way to format memory-strings (in shared memory, memory mapped files or any other memory segment) that can avoid all unneeded string copy and memory allocation/deallocations, while using all iostream facilities. [*Boost.Interprocess] *vectorstream* and *bufferstream* implement vector-based and fixed-size buffer based storage support for iostreams and all the formatting/locale hard work is done by standard `std::basic_streambuf<>` and `std::basic_iostream<>` classes. [section:vectorstream Formatting directly in your character vector: vectorstream] The *vectorstream* class family (*basic_vectorbuf*, *basic_ivectorstream* ,*basic_ovectorstream* and *basic_vectorstream*) is an efficient way to obtain formatted reading/writing directly in a character vector. This way, if a shared-memory vector is used, data is extracted/written from/to the shared-memory vector, without additional copy/allocation. We can see the declaration of basic_vectorstream here: //!A basic_iostream class that holds a character vector specified by CharVector //!template parameter as its formatting buffer. The vector must have //!contiguous storage, like std::vector, boost::interprocess::vector or //!boost::interprocess::basic_string template > class basic_vectorstream : public std::basic_iostream { public: typedef CharVector vector_type; typedef typename std::basic_ios ::char_type char_type; typedef typename std::basic_ios::int_type int_type; typedef typename std::basic_ios::pos_type pos_type; typedef typename std::basic_ios::off_type off_type; typedef typename std::basic_ios::traits_type traits_type; //!Constructor. Throws if vector_type default constructor throws. basic_vectorstream(std::ios_base::openmode mode = std::ios_base::in | std::ios_base::out); //!Constructor. Throws if vector_type(const Parameter ¶m) throws. template basic_vectorstream(const Parameter ¶m, std::ios_base::openmode mode = std::ios_base::in | std::ios_base::out); ~basic_vectorstream(){} //!Returns the address of the stored stream buffer. basic_vectorbuf* rdbuf() const; //!Swaps the underlying vector with the passed vector. //!This function resets the position in the stream. //!Does not throw. void swap_vector(vector_type &vect); //!Returns a const reference to the internal vector. //!Does not throw. const vector_type &vector() const; //!Preallocates memory from the internal vector. //!Resets the stream to the first position. //!Throws if the internals vector's memory allocation throws. void reserve(typename vector_type::size_type size); }; The vector type is templatized, so that we can use any type of vector: [*std::vector], [classref boost::interprocess::vector]... But the storage must be *contiguous*, we can't use a deque. We can even use *boost::interprocess::basic_string*, since it has a vector interface and it has contiguous storage. *We can't use std::string*, because although some std::string implementation are vector-based, others can have optimizations and reference-counted implementations. The user can obtain a const reference to the internal vector using `vector_type vector() const` function and he also can swap the internal vector with an external one calling `void swap_vector(vector_type &vect)`. The swap function resets the stream position. This functions allow efficient methods to obtain the formatted data avoiding all allocations and data copies. Let's see an example to see how to use vectorstream: [import ../example/doc_vectorstream.cpp] [doc_vectorstream] [endsect] [section:bufferstream Formatting directly in your character buffer: bufferstream] As seen, vectorstream offers an easy and secure way for efficient iostream formatting, but many times, we have to read or write formatted data from/to a fixed size character buffer (a static buffer, a c-string, or any other). Because of the overhead of stringstream, many developers (specially in embedded systems) choose sprintf family. The *bufferstream* classes offer iostream interface with direct formatting in a fixed size memory buffer with protection against buffer overflows. This is the interface: //!A basic_iostream class that uses a fixed size character buffer //!as its formatting buffer. template > class basic_bufferstream : public std::basic_iostream { public: // Typedefs typedef typename std::basic_ios ::char_type char_type; typedef typename std::basic_ios::int_type int_type; typedef typename std::basic_ios::pos_type pos_type; typedef typename std::basic_ios::off_type off_type; typedef typename std::basic_ios::traits_type traits_type; //!Constructor. Does not throw. basic_bufferstream(std::ios_base::openmode mode = std::ios_base::in | std::ios_base::out); //!Constructor. Assigns formatting buffer. Does not throw. basic_bufferstream(CharT *buffer, std::size_t length, std::ios_base::openmode mode = std::ios_base::in | std::ios_base::out); //!Returns the address of the stored stream buffer. basic_bufferbuf* rdbuf() const; //!Returns the pointer and size of the internal buffer. //!Does not throw. std::pair buffer() const; //!Sets the underlying buffer to a new value. Resets //!stream position. Does not throw. void buffer(CharT *buffer, std::size_t length); }; //Some typedefs to simplify usage typedef basic_bufferstream bufferstream; typedef basic_bufferstream wbufferstream; // ... While reading from a fixed size buffer, *bufferstream* activates endbit flag if we try to read an address beyond the end of the buffer. While writing to a fixed size buffer, *bufferstream* will active the badbit flag if a buffer overflow is going to happen and disallows writing. This way, the fixed size buffer formatting through *bufferstream* is secure and efficient, and offers a good alternative to sprintf/sscanf functions. Let's see an example: [import ../example/doc_bufferstream.cpp] [doc_bufferstream] As seen, *bufferstream* offers an efficient way to format data without any allocation and extra copies. This is very helpful in embedded systems, or formatting inside time-critical loops, where stringstream extra copies would be too expensive. Unlike sprintf/sscanf, it has protection against buffer overflows. As we know, according to the *Technical Report on C++ Performance*, it's possible to design efficient iostreams for embedded platforms, so this bufferstream class comes handy to format data to stack, static or shared memory buffers. [endsect] [endsect] [section:interprocess_smart_ptr Ownership smart pointers] C++ users know the importance of ownership smart pointers when dealing with resources. Boost offers a wide range of such type of pointers: `intrusive_ptr<>`, `scoped_ptr<>`, `shared_ptr<>`... When building complex shared memory/memory mapped files structures, programmers would like to use also the advantages of these smart pointers. The problem is that Boost and C++ TR1 smart pointers are not ready to be used for shared memory. The cause is that those smart pointers contain raw pointers and they use virtual functions, something that is not possible if you want to place your data in shared memory. The virtual function limitation makes even impossible to achieve the same level of functionality of Boost and TR1 with [*Boost.Interprocess] smart pointers. Interprocess ownership smart pointers are mainly "smart pointers containing smart pointers", so we can specify the pointer type they contain. [section:intrusive_ptr Intrusive pointer] [classref boost::interprocess::intrusive_ptr] is the generalization of `boost::intrusive_ptr<>` to allow non-raw pointers as intrusive pointer members. As the well-known `boost::intrusive_ptr` we must specify the pointee type but we also must also specify the pointer type to be stored in the intrusive_ptr: [c++] //!The intrusive_ptr class template stores a pointer to an object //!with an embedded reference count. intrusive_ptr is parameterized on //!T (the type of the object pointed to) and VoidPointer(a void pointer type //!that defines the type of pointer that intrusive_ptr will store). //!intrusive_ptr defines a class with a T* member whereas //!intrusive_ptr > defines a class with a offset_ptr member. //!Relies on unqualified calls to: //! //!void intrusive_ptr_add_ref(T * p); //!void intrusive_ptr_release(T * p); //! //!with (p != 0) //! //!The object is responsible for destroying itself. template class intrusive_ptr; So `boost::interprocess::intrusive_ptr` is equivalent to `boost::intrusive_ptr`. But if we want to place the intrusive_ptr in shared memory we must specify a relative pointer type like `boost::interprocess::intrusive_ptr >` [import ../example/doc_intrusive.cpp] [doc_intrusive] [endsect] [section:scoped_ptr Scoped pointer] `boost::interprocess::scoped_ptr<>` is the big brother of `boost::scoped_ptr<>`, which adds a custom deleter to specify how the pointer passed to the scoped_ptr must be destroyed. Also, the `pointer` typedef of the deleter will specify the pointer type stored by scoped_ptr. [c++] //!scoped_ptr stores a pointer to a dynamically allocated object. //!The object pointed to is guaranteed to be deleted, either on destruction //!of the scoped_ptr, or via an explicit reset. The user can avoid this //!deletion using release(). //!scoped_ptr is parameterized on T (the type of the object pointed to) and //!Deleter (the functor to be executed to delete the internal pointer). //!The internal pointer will be of the same pointer type as typename //!Deleter::pointer type (that is, if typename Deleter::pointer is //!offset_ptr, the internal pointer will be offset_ptr). template class scoped_ptr; `scoped_ptr<>` comes handy to implement *rollbacks* with exceptions: if an exception is thrown or we call `return` in the scope of `scoped_ptr<>` the deleter is automatically called so that *the deleter can be considered as a rollback* function. If all goes well, we call `release()` member function to avoid rollback when the `scoped_ptr` goes out of scope. [import ../example/doc_scoped_ptr.cpp] [doc_scoped_ptr] [endsect] [section:shared_ptr Shared pointer and weak pointer] [*Boost.Interprocess] also offers the possibility of creating non-intrusive reference-counted objects in managed shared memory or mapped files. Unlike [@http://www.boost.org/libs/smart_ptr/shared_ptr.htm boost::shared_ptr], due to limitations of mapped segments [classref boost::interprocess::shared_ptr] cannot take advantage of virtual functions to maintain the same shared pointer type while providing user-defined allocators and deleters. The allocator and the deleter are template parameters of the shared pointer. Since the reference count and other auxiliary data needed by [classref boost::interprocess::shared_ptr shared_ptr] must be created also in the managed segment, and the deleter has to delete the object from the segment, the user must specify an allocator object and a deleter object when constructing a non-empty instance of [classref boost::interprocess::shared_ptr shared_ptr], just like [*Boost.Interprocess] containers need to pass allocators in their constructors. Here is the declaration of [classref boost::interprocess::shared_ptr shared_ptr]: [c++] template class shared_ptr; * T is the type of the pointed type. * VoidAllocator is the allocator to be used to allocate auxiliary elements such as the reference count, the deleter... The internal `pointer` typedef of the allocator will determine the type of pointer that shared_ptr will internally use, so allocators defining `pointer` as `offset_ptr` will make all internal pointers used by `shared_ptr` to be also relative pointers. See [classref boost::interprocess::allocator] for a working allocator. * Deleter is the function object that will be used to destroy the pointed object when the last reference to the object is destroyed. The deleter functor will take a pointer to T of the same category as the void pointer defined by `VoidAllocator::pointer`. See [classref boost::interprocess::deleter] for a generic deleter that erases a object from a managed segment. With correctly specified parameters, [*Boost.Interprocess] users can create objects in shared memory that hold shared pointers pointing to other objects also in shared memory, obtaining the benefits of reference counting. Let's see how to create a shared pointer in a managed shared memory: [import ../example/doc_shared_ptr_explicit.cpp] [doc_shared_ptr_explicit] [classref boost::interprocess::shared_ptr] is very flexible and configurable (we can specify the allocator and the deleter, for example), but as shown the creation of a shared pointer in managed segments need too much typing. To simplify this usage, [classref boost::interprocess::shared_ptr] header offers a shared pointer definition helper class ([classref boost::interprocess::managed_shared_ptr managed_shared_ptr]) and a function ([funcref boost::interprocess::make_managed_shared_ptr make_managed_shared_ptr]) to easily construct a shared pointer from a type allocated in a managed segment with an allocator that will allocate the reference count also in the managed segment and a deleter that will erase the object from the segment. These utilities will use a [*Boost.Interprocess] allocator ([classref boost::interprocess::allocator]) and deleter ([classref boost::interprocess::deleter]) to do their job. The definition of the previous shared pointer could be simplified to the following: [c++] typedef managed_shared_ptr::type my_shared_ptr; And the creation of a shared pointer can be simplified to this: [c++] my_shared_ptr sh_ptr = make_managed_shared_ptr (segment.construct("object to share")(), segment); [*Boost.Interprocess] also offers a weak pointer named [classref boost::interprocess::weak_ptr weak_ptr] (with its corresponding [classref boost::interprocess::managed_weak_ptr managed_weak_ptr] and [funcref boost::interprocess::make_managed_weak_ptr make_managed_weak_ptr] utilities) to implement non-owning observers of an object owned by [classref boost::interprocess::shared_ptr shared_ptr]. Now let's see a detailed example of the use of [classref boost::interprocess::shared_ptr shared_ptr]: and [classref boost::interprocess::weak_ptr weak_ptr] [import ../example/doc_shared_ptr.cpp] [doc_shared_ptr] In general, using [*Boost.Interprocess]' [classref boost::interprocess::shared_ptr shared_ptr] and [classref boost::interprocess::weak_ptr weak_ptr] is very similar to their counterparts [@http://www.boost.org/libs/smart_ptr/shared_ptr.htm boost::shared_ptr] and [@http://www.boost.org/libs/smart_ptr/weak_ptr.htm boost::weak_ptr], but they need more template parameters and more run-time parameters in their constructors. Just like [@http://www.boost.org/libs/smart_ptr/shared_ptr.htm boost::shared_ptr] can be stored in a STL container, [classref boost::interprocess::shared_ptr shared_ptr] can also be stored in [*Boost.Interprocess] containers. If a programmer just uses [classref boost::interprocess::shared_ptr shared_ptr] to be able to insert dynamically constructed objects into a container constructed in the managed segment, but he does not need to share the ownership of that object with other objects [classref boost::interprocess::managed_unique_ptr managed_unique_ptr] is a much faster and easier to use alternative. [endsect] [section:unique_ptr Unique pointer] Unique ownership smart pointers are really useful to free programmers from manual resource liberation of non-shared objects. [*Boost.Interprocess]' `unique_ptr` is much like [classref boost::interprocess::scoped_ptr scoped_ptr] but it's [*moveable] and can be easily inserted in [*Boost.Interprocess] containers. Interprocess had its own `unique_ptr` implementation but from Boost 1.57, [*Boost.Interprocess] uses the improved and generic `boost::unique_ptr` implementation. Here is the declaration of the unique pointer class: [c++] template class unique_ptr; * T is the type of the object pointed by `unique_ptr`. * D is the deleter that will erase the object type of the object pointed by the unique_ptr when the unique pointer is destroyed (and if still has the ownership of the object). If the deleter defines an internal `pointer` typedef, `unique_ptr`] will use an internal pointer of the same type. So if `D::pointer` is `offset_ptr` the unique pointer will store a relative pointer instead of a raw one. This allows placing `unique_ptr` in shared memory and memory-mapped files. `unique_ptr` can release the ownership of the stored pointer so it's useful also to be used as a rollback function. One of the main properties of the class is that [*is not copyable, but only moveable]. When a unique pointer is moved to another one, the ownership of the pointer is transferred from the source unique pointer to the target unique pointer. If the target unique pointer owned an object, that object is first deleted before taking ownership of the new object. [*Boost.Interprocess] also offers auxiliary types to easily define and construct unique pointers that can be placed in managed segments and will correctly delete owned object from the segment: [classref boost::interprocess::managed_unique_ptr managed_unique_ptr] and [funcref boost::interprocess::make_managed_unique_ptr make_managed_unique_ptr] utilities. Here we see an example of the use `unique_ptr` including creating containers of such objects: [import ../example/doc_unique_ptr.cpp] [doc_unique_ptr] [endsect] [endsect] [section:architecture Architecture and internals] [section:basic_guidelines Basic guidelines] When building [*Boost.Interprocess] architecture, I took some basic guidelines that can be summarized by these points: * [*Boost.Interprocess] should be portable at least in UNIX and Windows systems. That means unifying not only interfaces but also behaviour. This is why [*Boost.Interprocess] has chosen kernel or filesystem persistence for shared memory and named synchronization mechanisms. Process persistence for shared memory is also desirable but it's difficult to achieve in UNIX systems. * [*Boost.Interprocess] inter-process synchronization primitives should be equal to thread synchronization primitives. [*Boost.Interprocess] aims to have an interface compatible with the C++ standard thread API. * [*Boost.Interprocess] architecture should be modular, customizable but efficient. That's why [*Boost.Interprocess] is based on templates and memory algorithms, index types, mutex types and other classes are templatizable. * [*Boost.Interprocess] architecture should allow the same concurrency as thread based programming. Different mutual exclusion levels are defined so that a process can concurrently allocate raw memory when expanding a shared memory vector while another process can be safely searching a named object. * [*Boost.Interprocess] containers know nothing about [*Boost.Interprocess]. All specific behaviour is contained in the STL-like allocators. That allows STL vendors to slightly modify (or better said, generalize) their standard container implementations and obtain a fully std::allocator and boost::interprocess::allocator compatible container. This also make [*Boost.Interprocess] containers compatible with standard algorithms. [*Boost.Interprocess] is built above 3 basic classes: a [*memory algorithm], a [*segment manager] and a [*managed memory segment]: [endsect] [section:architecture_algorithm_to_managed From the memory algorithm to the managed segment] [section:architecture_memory_algorithm The memory algorithm] The [*memory algorithm] is an object that is placed in the first bytes of a shared memory/memory mapped file segment. The [*memory algorithm] can return portions of that segment to users marking them as used and the user can return those portions to the [*memory algorithm] so that the [*memory algorithm] mark them as free again. There is an exception though: some bytes beyond the end of the memory algorithm object, are reserved and can't be used for this dynamic allocation. This "reserved" zone will be used to place other additional objects in a well-known place. To sum up, a [*memory algorithm] has the same mission as malloc/free of standard C library, but it just can return portions of the segment where it is placed. The layout of a memory segment would be: [c++] Layout of the memory segment: ____________ __________ ____________________________________________ | | | | | memory | reserved | The memory algorithm will return portions | | algorithm | | of the rest of the segment. | |____________|__________|____________________________________________| The [*memory algorithm] takes care of memory synchronizations, just like malloc/free guarantees that two threads can call malloc/free at the same time. This is usually achieved placing a process-shared mutex as a member of the memory algorithm. Take in care that the memory algorithm knows [*nothing] about the segment (if it is shared memory, a shared memory file, etc.). For the memory algorithm the segment is just a fixed size memory buffer. The [*memory algorithm] is also a configuration point for the rest of the [*Boost.Interprocess] framework since it defines two basic types as member typedefs: [c++] typedef /*implementation dependent*/ void_pointer; typedef /*implementation dependent*/ mutex_family; The `void_pointer` typedef defines the pointer type that will be used in the [*Boost.Interprocess] framework (segment manager, allocators, containers). If the memory algorithm is ready to be placed in a shared memory/mapped file mapped in different base addresses, this pointer type will be defined as `offset_ptr` or a similar relative pointer. If the [*memory algorithm] will be used just with fixed address mapping, `void_pointer` can be defined as `void*`. The rest of the interface of a [*Boost.Interprocess] [*memory algorithm] is described in [link interprocess.customizing_interprocess.custom_interprocess_alloc Writing a new shared memory allocation algorithm] section. As memory algorithm examples, you can see the implementations [classref boost::interprocess::simple_seq_fit simple_seq_fit] or [classref boost::interprocess::rbtree_best_fit rbtree_best_fit] classes. [endsect] [section:architecture_segment_manager The segment manager] The *segment manager*, is an object also placed in the first bytes of the managed memory segment (shared memory, memory mapped file), that offers more sophisticated services built above the [*memory algorithm]. How can [*both] the segment manager and memory algorithm be placed in the beginning of the segment? That's because the segment manager [*owns] the memory algorithm: The truth is that the memory algorithm is [*embedded] in the segment manager: [c++] The layout of managed memory segment: _______ _________________ | | | | | some | memory | other |<- The memory algorithm considers |members|algorithm|members| "other members" as reserved memory, so |_______|_________|_______| it does not use it for dynamic allocation. |_________________________|____________________________________________ | | | | segment manager | The memory algorithm will return portions | | | of the rest of the segment. | |_________________________|____________________________________________| The [*segment manager] initializes the memory algorithm and tells the memory manager that it should not use the memory where the rest of the [*segment manager]'s member are placed for dynamic allocations. The other members of the [*segment manager] are [*a recursive mutex] (defined by the memory algorithm's [*mutex_family::recursive_mutex] typedef member), and [*two indexes (maps)]: one to implement named allocations, and another one to implement "unique instance" allocations. * The first index is a map with a pointer to a c-string (the name of the named object) as a key and a structure with information of the dynamically allocated object (the most important being the address and the size of the object). * The second index is used to implement "unique instances" and is basically the same as the first index, but the name of the object comes from a `typeid(T).name()` operation. The memory needed to store [name pointer, object information] pairs in the index is allocated also via the *memory algorithm*, so we can tell that internal indexes are just like ordinary user objects built in the segment. The rest of the memory to store the name of the object, the object itself, and meta-data for destruction/deallocation is allocated using the *memory algorithm* in a single `allocate()` call. As seen, the [*segment manager] knows [*nothing] about shared memory/memory mapped files. The [*segment manager] itself does not allocate portions of the segment, it just asks the *memory algorithm* to allocate the needed memory from the rest of the segment. The [*segment manager] is a class built above the memory algorithm that offers named object construction, unique instance constructions, and many other services. The [*segment manager] is implemented in [*Boost.Interprocess] by the [classref boost::interprocess::segment_manager segment_manager] class. [c++] template class IndexType> class segment_manager; As seen, the segment manager is quite generic: we can specify the character type to be used to identify named objects, we can specify the memory algorithm that will control dynamically the portions of the memory segment, and we can specify also the index type that will store the [name pointer, object information] mapping. We can construct our own index types as explained in [link interprocess.customizing_interprocess.custom_indexes Building custom indexes] section. [endsect] [section:architecture_managed_memory Boost.Interprocess managed memory segments] The [*Boost.Interprocess] managed memory segments that construct the shared memory/memory mapped file, place there the segment manager and forward the user requests to the segment manager. For example, [classref boost::interprocess::basic_managed_shared_memory basic_managed_shared_memory] is a [*Boost.Interprocess] managed memory segment that works with shared memory. [classref boost::interprocess::basic_managed_mapped_file basic_managed_mapped_file] works with memory mapped files, etc... Basically, the interface of a [*Boost.Interprocess] managed memory segment is the same as the [*segment manager] but it also offers functions to "open", "create", or "open or create" shared memory/memory-mapped files segments and initialize all needed resources. Managed memory segment classes are not built in shared memory or memory mapped files, they are normal C++ classes that store a pointer to the segment manager (which is built in shared memory or memory mapped files). Apart from this, managed memory segments offer specific functions: `managed_mapped_file` offers functions to flush memory contents to the file, `managed_heap_memory` offers functions to expand the memory, etc... Most of the functions of [*Boost.Interprocess] managed memory segments can be shared between all managed memory segments, since many times they just forward the functions to the segment manager. Because of this, in [*Boost.Interprocess] all managed memory segments derive from a common class that implements memory-independent (shared memory, memory mapped files) functions: [@../../boost/interprocess/detail/managed_memory_impl.hpp boost::interprocess::ipcdetail::basic_managed_memory_impl] Deriving from this class, [*Boost.Interprocess] implements several managed memory classes, for different memory backends: * [classref boost::interprocess::basic_managed_shared_memory basic_managed_shared_memory] (for shared memory). * [classref boost::interprocess::basic_managed_mapped_file basic_managed_mapped_file] (for memory mapped files). * [classref boost::interprocess::basic_managed_heap_memory basic_managed_heap_memory] (for heap allocated memory). * [classref boost::interprocess::basic_managed_external_buffer basic_managed_external_buffer] (for user provided external buffer). [endsect] [endsect] [section:allocators_containers Allocators and containers] [section:allocators Boost.Interprocess allocators] The [*Boost.Interprocess] STL-like allocators are fairly simple and follow the usual C++ allocator approach. Normally, allocators for STL containers are based above new/delete operators and above those, they implement pools, arenas and other allocation tricks. In [*Boost.Interprocess] allocators, the approach is similar, but all allocators are based on the *segment manager*. The segment manager is the only one that provides from simple memory allocation to named object creations. [*Boost.Interprocess] allocators always store a pointer to the segment manager, so that they can obtain memory from the segment or share a common pool between allocators. As you can imagine, the member pointers of the allocator are not a raw pointers, but pointer types defined by the `segment_manager::void_pointer` type. Apart from this, the `pointer` typedef of [*Boost.Interprocess] allocators is also of the same type of `segment_manager::void_pointer`. This means that if our allocation algorithm defines `void_pointer` as `offset_ptr`, `boost::interprocess::allocator` will store an `offset_ptr` to point to the segment manager and the `boost::interprocess::allocator::pointer` type will be `offset_ptr`. This way, [*Boost.Interprocess] allocators can be placed in the memory segment managed by the segment manager, that is, shared memory, memory mapped files, etc... [endsect] [section:implementation_segregated_storage_pools Implementation of [*Boost.Interprocess] segregated storage pools] Segregated storage pools are simple and follow the classic segregated storage algorithm. * The pool allocates chunks of memory using the segment manager's raw memory allocation functions. * The chunk contains a pointer to form a singly linked list of chunks. The pool will contain a pointer to the first chunk. * The rest of the memory of the chunk is divided in nodes of the requested size and no memory is used as payload for each node. Since the memory of a free node is not used that memory is used to place a pointer to form a singly linked list of free nodes. The pool has a pointer to the first free node. * Allocating a node is just taking the first free node from the list. If the list is empty, a new chunk is allocated, linked in the list of chunks and the new free nodes are linked in the free node list. * Deallocation returns the node to the free node list. * When the pool is destroyed, the list of chunks is traversed and memory is returned to the segment manager. The pool is implemented by the [@../../boost/interprocess/allocators/detail/node_pool.hpp private_node_pool and shared_node_pool] classes. [endsect] [section:implementation_adaptive_pools Implementation of [*Boost.Interprocess] adaptive pools] Adaptive pools are a variation of segregated lists but they have a more complicated approach: * Instead of using raw allocation, the pool allocates [*aligned] chunks of memory using the segment manager. This is an [*essential] feature since a node can reach its chunk information applying a simple mask to its address. * The chunks contains pointers to form a doubly linked list of chunks and an additional pointer to create a singly linked list of free nodes placed on that chunk. So unlike the segregated storage algorithm, the free list of nodes is implemented [*per chunk]. * The pool maintains the chunks in increasing order of free nodes. This improves locality and minimizes the dispersion of node allocations across the chunks facilitating the creation of totally free chunks. * The pool has a pointer to the chunk with the minimum (but not zero) free nodes. This chunk is called the "active" chunk. * Allocating a node is just returning the first free node of the "active" chunk. The list of chunks is reordered according to the free nodes count. The pointer to the "active" pool is updated if necessary. * If the pool runs out of nodes, a new chunk is allocated, and pushed back in the list of chunks. The pointer to the "active" pool is updated if necessary. * Deallocation returns the node to the free node list of its chunk and updates the "active" pool accordingly. * If the number of totally free chunks exceeds the limit, chunks are returned to the segment manager. * When the pool is destroyed, the list of chunks is traversed and memory is returned to the segment manager. The adaptive pool is implemented by the [@../../boost/interprocess/allocators/detail/adaptive_node_pool.hpp private_adaptive_node_pool and adaptive_node_pool] classes. [endsect] [section:architecture_containers Boost.Interprocess containers] [*Boost.Interprocess] containers are standard conforming counterparts of STL containers in `boost::interprocess` namespace, but with these little details: * [*Boost.Interprocess] STL containers don't assume that memory allocated with an allocator can be deallocated with other allocator of the same type. They always compare allocators with `operator==()` to know if this is possible. * The pointers of the internal structures of the [*Boost.Interprocess] containers are of the same type the `pointer` type defined by the allocator of the container. This allows placing containers in managed memory segments mapped in different base addresses. [endsect] [endsect] [section:performance Performance of Boost.Interprocess] This section tries to explain the performance characteristics of [*Boost.Interprocess], so that you can optimize [*Boost.Interprocess] usage if you need more performance. [section:performance_allocations Performance of raw memory allocations] You can have two types of raw memory allocations with [*Boost.Interprocess] classes: * [*Explicit]: The user calls `allocate()` and `deallocate()` functions of managed_shared_memory/managed_mapped_file... managed memory segments. This call is translated to a `MemoryAlgorithm::allocate()` function, which means that you will need just the time that the memory algorithm associated with the managed memory segment needs to allocate data. * [*Implicit]: For example, you are using `boost::interprocess::allocator<...>` with [*Boost.Interprocess] containers. This allocator calls the same `MemoryAlgorithm::allocate()` function than the explicit method, [*every] time a vector/string has to reallocate its buffer or [*every] time you insert an object in a node container. If you see that memory allocation is a bottleneck in your application, you have these alternatives: * If you use map/set associative containers, try using `flat_map` family instead of the map family if you mainly do searches and the insertion/removal is mainly done in an initialization phase. The overhead is now when the ordered vector has to reallocate its storage and move data. You can also call the `reserve()` method of these containers when you know beforehand how much data you will insert. However in these containers iterators are invalidated in insertions so this substitution is only effective in some applications. * Use a [*Boost.Interprocess] pooled allocator for node containers, because pooled allocators call `allocate()` only when the pool runs out of nodes. This is pretty efficient (much more than the current default general-purpose algorithm) and this can save a lot of memory. See [link interprocess.allocators_containers.stl_allocators_segregated_storage Segregated storage node allocators] and [link interprocess.allocators_containers.stl_allocators_adaptive Adaptive node allocators] for more information. * Write your own memory algorithm. If you have experience with memory allocation algorithms and you think another algorithm is better suited than the default one for your application, you can specify it in all [*Boost.Interprocess] managed memory segments. See the section [link interprocess.customizing_interprocess.custom_interprocess_alloc Writing a new shared memory allocation algorithm] to know how to do this. If you think its better than the default one for general-purpose applications, be polite and donate it to [*Boost.Interprocess] to make it default! [endsect] [section:performance_named_allocation Performance of named allocations] [*Boost.Interprocess] allows the same parallelism as two threads writing to a common structure, except when the user creates/searches named/unique objects. The steps when creating a named object are these: * Lock a recursive mutex (so that you can make named allocations inside the constructor of the object to be created). * Try to insert the [name pointer, object information] in the name/object index. This lookup has to assure that the name has not been used before. This is achieved calling `insert()` function in the index. So the time this requires is dependent on the index type (ordered vector, tree, hash...). This can require a call to the memory algorithm allocation function if the index has to be reallocated, it's a node allocator, uses pooled allocations... * Allocate a single buffer to hold the name of the object, the object itself, and meta-data for destruction (number of objects, etc...). * Call the constructors of the object being created. If it's an array, one constructor per array element. * Unlock the recursive mutex. The steps when destroying a named object using the name of the object (`destroy(name)`) are these: * Lock a recursive mutex . * Search in the index the entry associated to that name. Copy that information and erase the index entry. This is done using `find(const key_type &)` and `erase(iterator)` members of the index. This can require element reordering if the index is a balanced tree, an ordered vector... * Call the destructor of the object (many if it's an array). * Deallocate the memory buffer containing the name, metadata and the object itself using the allocation algorithm. * Unlock the recursive mutex. The steps when destroying a named object using the pointer of the object (`destroy_ptr(T *ptr)`) are these: * Lock a recursive mutex . * Depending on the index type, this can be different: * If the index is a node index, (marked with `boost::interprocess::is_node_index` specialization): Take the iterator stored near the object and call `erase(iterator)`. This can require element reordering if the index is a balanced tree, an ordered vector... * If it's not an node index: Take the name stored near the object and erase the index entry calling `erase(const key &). This can require element reordering if the index is a balanced tree, an ordered vector... * Call the destructor of the object (many if it's an array). * Deallocate the memory buffer containing the name, metadata and the object itself using the allocation algorithm. * Unlock the recursive mutex. If you see that the performance is not good enough you have these alternatives: * Maybe the problem is that the lock time is too big and it hurts parallelism. Try to reduce the number of named objects in the global index and if your application serves several clients try to build a new managed memory segment for each one instead of using a common one. * Use another [*Boost.Interprocess] index type if you feel the default one is not fast enough. If you are not still satisfied, write your own index type. See [link interprocess.customizing_interprocess.custom_indexes Building custom indexes] for this. * Destruction via pointer is at least as fast as using the name of the object and can be faster (in node containers, for example). So if your problem is that you make at lot of named destructions, try to use the pointer. If the index is a node index you can save some time. [endsect] [endsect] [endsect] [section:customizing_interprocess Customizing Boost.Interprocess] [section:custom_interprocess_alloc Writing a new shared memory allocation algorithm] If the default algorithm does not satisfy user requirements, it's easy to provide different algorithms like bitmapping or more advanced segregated lists to meet requirements. The class implementing the algorithm must be compatible with shared memory, so it shouldn't have any virtual function or virtual inheritance or any indirect base class with virtual function or inheritance. This is the interface to be implemented: [c++] class my_algorithm { public: //!The mutex type to be used by the rest of Interprocess framework typedef implementation_defined mutex_family; //!The pointer type to be used by the rest of Interprocess framework typedef implementation_defined void_pointer; //!Constructor. "size" is the total size of the managed memory segment, //!"extra_hdr_bytes" indicates the extra bytes after the sizeof(my_algorithm) //!that the allocator should not use at all. my_algorithm (std::size_t size, std::size_t extra_hdr_bytes); //!Obtains the minimum size needed by the algorithm static std::size_t get_min_size (std::size_t extra_hdr_bytes); //!Allocates bytes, returns 0 if there is not more memory void* allocate (std::size_t nbytes); //!Deallocates previously allocated bytes void deallocate (void *adr); //!Returns the size of the memory segment std::size_t get_size() const; //!Increases managed memory in extra_size bytes more void grow(std::size_t extra_size); /*...*/ }; Let's see the public typedefs to define: [c++] typedef /* . . . */ void_pointer; typedef /* . . . */ mutex_family; The `void_pointer` typedef specifies the pointer type to be used in the [*Boost.Interprocess] framework that uses the algorithm. For example, if we define [c++] typedef void * void_pointer; all [*Boost.Interprocess] framework using this algorithm will use raw pointers as members. But if we define: [c++] typedef offset_ptr void_pointer; then all [*Boost.Interprocess] framework will use relative pointers. The `mutex_family` is a structure containing typedefs for different interprocess_mutex types to be used in the [*Boost.Interprocess] framework. For example the defined [c++] struct mutex_family { typedef boost::interprocess::interprocess_mutex mutex_type; typedef boost::interprocess::interprocess_recursive_mutex recursive_mutex_type; }; defines all interprocess_mutex types using boost::interprocess interprocess_mutex types. The user can specify the desired mutex family. [c++] typedef mutex_family mutex_family; The new algorithm (let's call it *my_algorithm*) must implement all the functions that boost::interprocess::rbtree_best_fit class offers: * [*my_algorithm]'s constructor must take 2 arguments: * [*size] indicates the total size of the managed memory segment, and [*my_algorithm] object will be always constructed a at offset 0 of the memory segment. * The [*extra_hdr_bytes] parameter indicates the number of bytes after the offset `sizeof(my_algorithm)` that [*my_algorithm] can't use at all. This extra bytes will be used to store additional data that should not be overwritten. So, [*my_algorithm] will be placed at address XXX of the memory segment, and will manage the [*[XXX + sizeof(my_algorithm) + extra_hdr_bytes, XXX + size)] range of the segment. * The [*get_min_size()] function should return the minimum space the algorithm needs to be valid with the passed [*extra_hdr_bytes] parameter. This function will be used to check if the memory segment is big enough to place the algorithm there. * The [*allocate()] function must return 0 if there is no more available memory. The memory returned by [*my_algorithm] must be aligned to the most restrictive memory alignment of the system. This function should be executed with the synchronization capabilities offered by `typename mutex_family::mutex_type` interprocess_mutex. That means, that if we define `typedef mutex_family mutex_family;` then this function should offer the same synchronization as if it was surrounded by an interprocess_mutex lock/unlock. Normally, this is implemented using a member of type `mutex_family::mutex_type`, but it could be done using atomic instructions or lock free algorithms. * The [*deallocate()] function must make the returned buffer available for new allocations. This function should offer the same synchronization as `allocate()`. * The [*size()] function will return the passed [*size] parameter in the constructor. So, [*my_algorithm] should store the size internally. * The [*grow()] function will expand the managed memory by [*my_algorithm] in [*extra_size] bytes. So [*size()] function should return the updated size, and the new managed memory range will be (if the address where the algorithm is constructed is XXX): [*[XXX + sizeof(my_algorithm) + extra_hdr_bytes, XXX + old_size + extra_size)]. This function should offer the same synchronization as `allocate()`. That's it. Now we can create new managed shared memory that uses our new algorithm: [c++] //Managed memory segment to allocate named (c-string) objects //using a user-defined memory allocation algorithm basic_managed_shared_memory my_managed_shared_memory; [endsect] [section:custom_allocators Building custom STL compatible allocators for Boost.Interprocess] If provided STL-like allocators don't satisfy user needs, the user can implement another STL compatible allocator using raw memory allocation and named object construction functions. The user can this way implement more suitable allocation schemes on top of basic shared memory allocation schemes, just like more complex allocators are built on top of new/delete functions. When using a managed memory segment, [*get_segment_manager()] function returns a pointer to the segment manager. With this pointer, the raw memory allocation and named object construction functions can be called directly: [c++] //Create the managed shared memory and initialize resources managed_shared_memory segment (create_only ,"/MySharedMemory" //segment name ,65536); //segment size in bytes //Obtain the segment manager managed_shared_memory::segment_manager *segment_mngr = segment.get_segment_manager(); //With the segment manager, now we have access to all allocation functions segment_mngr->deallocate(segment_mngr->allocate(32)); segment_mngr->construct("My_Int")[32](0); segment_mngr->destroy("My_Int"); //Initialize the custom, managed memory segment compatible //allocator with the segment manager. // //MySTLAllocator uses segment_mngr->xxx functions to //implement its allocation scheme MySTLAllocator stl_alloc(segment_mngr); //Alias a new vector type that uses the custom STL compatible allocator typedef std::vector > MyVect; //Construct the vector in shared memory with the allocator as constructor parameter segment.construct("MyVect_instance")(stl_alloc); The user can create new STL compatible allocators that use the segment manager to access to all memory management/object construction functions. All [*Boost.Interprocess]' STL compatible allocators are based on this approach. [*Remember] that to be compatible with managed memory segments, allocators should define their *pointer* typedef as the same pointer family as `segment_manager::void_pointer` typedef. This means that if `segment_manager::void_pointer` is `offset_ptr`, `MySTLAllocator` should define `pointer` as `offset_ptr`. The reason for this is that allocators are members of containers, and if we want to put the container in a managed memory segment, the allocator should be ready for that. [endsect] [section:custom_indexes Building custom indexes] The managed memory segment uses a name/object index to speed up object searching and creation. Default specializations of managed memory segments (`managed_shared_memory` for example), use `boost::interprocess::flat_map` as index. However, the index type can be chosen via template parameter, so that the user can define its own index type if he needs that. To construct a new index type, the user must create a class with the following guidelines: * The interface of the index must follow the common public interface of std::map and std::tr1::unordered_map including public typedefs. The `value_type` typedef can be of type: [c++] std::pair or [c++] std::pair so that ordered arrays or deques can be used as index types. Some known classes following this basic interface are `boost::unordered_map`, `boost::interprocess::flat_map` and `boost::interprocess::map`. * The class must be a class template taking only a traits struct of this type: [c++] struct index_traits { typedef /*...*/ key_type; typedef /*...*/ mapped_type; typedef /*...*/ segment_manager; }; [c++] template class my_index_type; The `key_type` typedef of the passed `index_traits` will be a specialization of the following class: [c++] //!The key of the named allocation information index. Stores a to //!a null string and the length of the string to speed up sorting template<...> struct index_key { typedef /*...*/ char_type; typedef /*...*/ const_char_ptr_t; //Pointer to the object's name (null terminated) const_char_ptr_t mp_str; //Length of the name buffer (null NOT included) std::size_t m_len; //!Constructor of the key index_key (const CharT *name, std::size_t length); //!Less than function for index ordering bool operator < (const index_key & right) const; //!Equal to function for index ordering bool operator == (const index_key & right) const; }; The `mapped_type` is not directly modified by the customized index but it is needed to define the index type. The *segment_manager* will be the type of the segment manager that will manage the index. `segment_manager` will define interesting internal types like `void_pointer` or `mutex_family`. * The constructor of the customized index type must take a pointer to segment_manager as constructor argument: [c++] constructor(segment_manager *segment_mngr); * The index must provide a memory reservation function, that optimizes the index if the user knows the number of elements to be inserted in the index: [c++] void reserve(std::size_t n); For example, the index type `flat_map_index` based in `boost::interprocess::flat_map` is just defined as: [import ../../../boost/interprocess/indexes/flat_map_index.hpp] [flat_map_index] If the user is defining a node container based index (a container whose iterators are not invalidated when inserting or erasing other elements), [*Boost.Interprocess] can optimize named object destruction when destructing via pointer. [*Boost.Interprocess] can store an iterator next to the object and instead of using the name of the object to erase the index entry, it uses the iterator, which is a faster operation. So if you are creating a new node container based index (for example, a tree), you should define an specialization of `boost::interprocess::is_node_index<...>` defined in ``: [c++] //!Trait classes to detect if an index is a node //!index. This allows more efficient operations //!when deallocating named objects. template struct is_node_index > { static const bool value = true; }; Interprocess also defines other index types: * [*boost::map_index] uses *boost::interprocess::map* as index type. * [*boost::null_index] that uses an dummy index type if the user just needs anonymous allocations and wants to save some space and class instantiations. Defining a new managed memory segment that uses the new index is easy. For example, a new managed shared memory that uses the new index: [c++] //!Defines a managed shared memory with a c-strings as //!a keys, the red-black tree best fit algorithm (with process-shared mutexes //!and offset_ptr pointers) as raw shared memory management algorithm //!and a custom index typedef basic_managed_shared_memory < char, rbtree_best_fit, my_index_type > my_managed_shared_memory; [endsect] [endsect] [section:acknowledgements_notes Acknowledgements, notes and links] [section:notes_windows Notes for Windows users] [section:notes_windows_com_init COM Initialization] [*Boost.Interprocess] uses the Windows COM library to implement some features and initializes it with concurrency model `COINIT_APARTMENTTHREADED`. If the COM library was already initialized by the calling thread for another concurrency model, [*Boost.Interprocess] handles this gracefully and uses COM calls for the already initialized model. If for some reason, you want [*Boost.Interprocess] to initialize the COM library with another model, define the macro `BOOST_INTERPROCESS_WINDOWS_COINIT_MODEL` before including [*Boost.Interprocess] to one of these values: * `COINIT_APARTMENTTHREADED_BIPC` * `COINIT_MULTITHREADED_BIPC` * `COINIT_DISABLE_OLE1DDE_BIPC` * `COINIT_SPEED_OVER_MEMORY_BIPC` [endsect] [section:notes_windows_shm_folder Shared memory emulation folder] Shared memory (`shared_memory_object`) is implemented in Windows using memory mapped files, placed in a shared directory in the shared documents folder (`SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\Common AppData`). This directory name is the last bootup time obtained via COM calls (if `BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME`) defined or searching the system log for a startup event (the default implementation), so that each bootup shared memory is created in a new folder obtaining kernel persistence shared memory. If using `BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME`, due to COM implementation related errors, in Boost 1.48 & Boost 1.49 the bootup-time folder was dumped and files were directly created in shared documents folder, reverting to filesystem persistence shared memory. Boost 1.50 fixed those issues and recovered bootup time directory and kernel persistence. If you need to reproduce Boost 1.48 & Boost 1.49 behaviour to communicate with applications compiled with that version, comment `#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME` directive in the Windows configuration part of `boost/interprocess/detail/workaround.hpp`. If using the default implementation, (`BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME` undefined) and the Startup Event is not found, this might be due to some buggy software that floods or erases the event log. In any error case (shared documents folder is not defined or bootup time could not be obtained, the library throws an error. You still can use [*Boost.Interprocess] defining your own directory as the shared directory. When your shared directory is a compile-time constant, define `BOOST_INTERPROCESS_SHARED_DIR_PATH` when using the library and that path will be used to place shared memory files. When you have to determine the shared directory at runtime, define `BOOST_INTERPROCESS_SHARED_DIR_FUNC` and implement the function [c++] namespace boost { namespace interprocess { namespace ipcdetail { void get_shared_dir(std::string &shared_dir); } } } [endsect] [section:boost_use_windows_h BOOST_USE_WINDOWS_H support] If `BOOST_USE_WINDOWS_H` is defined, and other windows SDK files are included, otherwise the library declares needed functions and structures to reduce the impact of including those heavy headers. [endsect] [endsect] [section:notes_linux Notes for Linux users] [section:notes_linux_shm_folder Shared memory emulation folder] On systems without POSIX shared memory support, shared memory objects are implemented as memory mapped files, using a directory placed in "/tmp" that can include (if `BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME` is defined) the last bootup time (if the OS supports it). As in Windows, in any error case obtaining this directory the library throws an error . When your shared directory is a compile-time constant, define `BOOST_INTERPROCESS_SHARED_DIR_PATH` when using the library and that path will be used to place shared memory files. When you have to determine the shared directory at runtime, define `BOOST_INTERPROCESS_SHARED_DIR_FUNC` and implement the function [c++] namespace boost { namespace interprocess { namespace ipcdetail { void get_shared_dir(std::string &shared_dir); } } } [endsect] [section:notes_linux_overcommit Overcommit] The committed address space is the total amount of virtual memory (swap or physical memory/RAM) that the kernel might have to supply if all applications decide to access all of the memory they've requested from the kernel. By default, Linux allows processes to commit more virtual memory than available in the system. If that memory is not accessed, no physical memory + swap is actually used. The reason for this behaviour is that Linux tries to optimize memory usage on forked processes; fork() creates a full copy of the process space, but with overcommitted memory, in this new forked instance only pages which have been written to actually need to be allocated by the kernel. If applications access more memory than available, then the kernel must free memory in the hard way: the OOM (Out Of Memory)-killer picks some processes to kill in order to recover memory. [*Boost.Interprocess] has no way to change this behaviour and users might suffer the OOM-killer when accessing shared memory. According to the [@http://www.kernel.org/doc/Documentation/vm/overcommit-accounting Kernel documentation], the Linux kernel supports several overcommit modes. If you need non-kill guarantees in your application, you should change this overcommit behaviour. [endsect] [endsect] [section:thanks_to Thanks to...] Many people have contributed with ideas and revisions, so this is the place to thank them: * Thanks to all people who have shown interest in the library and have downloaded and tested the snapshots. * Thanks to [*Francis Andre] and [*Anders Hybertz] for their ideas and suggestions. Many of them are not implemented yet but I hope to include them when library gets some stability. * Thanks to [*Matt Doyle], [*Steve LoBasso], [*Glenn Schrader], [*Hiang Swee Chiang], [*Phil Endecott], [*Rene Rivera], [*Harold Pirtle], [*Paul Ryan], [*Shumin Wu], [*Michal Wozniak], [*Peter Johnson], [*Alex Ott], [*Shane Guillory], [*Steven Wooding] and [*Kim Barrett] for their bug fixes and library testing. * Thanks to [*Martin Adrian] who suggested the use of Interprocess framework for user defined buffers. * Thanks to [*Synge Todo] for his boostbook-doxygen patch to improve Interprocess documentation. * Thanks to [*Olaf Krzikalla] for his Intrusive library. I have taken some ideas to improve red black tree implementation from his library. * Thanks to [*Daniel James] for his unordered_map/set family and his help with allocators. His great unordered implementation has been a reference to design exception safe containers. * Thanks to [*Howard Hinnant] for his amazing help, specially explaining allocator swapping, move semantics and for developing upgradable mutex and lock transfer features. * Thanks to [*Pavel Vozenilek] for his continuous review process, suggestions, code and help. He is the major supporter of Interprocess library. The library has grown with his many and great advices. * And finally, thank you to all Boosters. [*Long live to C++!] [endsect] [section:release_notes Release Notes] [section:release_notes_boost_1_71_00 Boost 1.71 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/85 GitHub #85 (['"warning: Implicit conversion loses integer precision"])]. * [@https://github.com/boostorg/interprocess/pull/86 GitHub #86 (['"warning: Possible misuse of comma operator"])]. [endsect] [section:release_notes_boost_1_70_00 Boost 1.70 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/51 GitHub Pull #51 (['"United handling of wait_for_single_object"])]. * [@https://github.com/boostorg/interprocess/pull/78 GitHub Pull #78 (['"Fix -Wextra-semi clang warnings"])]. [endsect] [section:release_notes_boost_1_69_00 Boost 1.69 Release] * Deprecated GCC < 4.3 and MSVC < 9.0 (Visual 2008) compilers. * Fixed bugs: * [@https://github.com/boostorg/interprocess/issues/59 GitHub Issue #59 (['"warning: ISO C++ prohibits anonymous structs [-Wpedantic]"])]. * [@https://github.com/boostorg/interprocess/issues/60 GitHub Issue #60 (['"warning: cast between incompatible function types from boost::interprocess::winapi::farproc_t..."])]. * [@https://github.com/boostorg/interprocess/issues/61 GitHub Issue #61 (['"warning: struct winapi::*_BIPC has virtual functions and accessible non-virtual destructor"])]. * [@https://github.com/boostorg/interprocess/issues/64 GitHub Issue #64 (['"UBSan: runtime error: load of value 4294967295, (...) for type 'boost::interprocess::mode_t'"])]. * [@https://github.com/boostorg/interprocess/pull/68 GitHub Pull #68 (['"Prepare for C++20 and remove "throw()" usage"])]. * [@https://github.com/boostorg/interprocess/pull/70 GitHub Pull #70 (['"Fix deadlock in named_condition::notify_one"])]. [endsect] [section:release_notes_boost_1_68_00 Boost 1.68 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/issues/53 GitHub Issue #53 (['"Crash waiting on condition variable from multiple processes in do_wait()"])]. * [@https://github.com/boostorg/interprocess/issues/54 GitHub Issue #54 (['"fill_system_message() ignores an error returned by winapi::format_message()"])]. [endsect] [section:release_notes_boost_1_67_00 Boost 1.67 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/45 GitHub Pull #45 (['"Make intrusive_ptr move constructible/assignable"])]. * [@https://github.com/boostorg/interprocess/pull/48 GitHub Pull #48 (['"Win32: Fix read of reg_expand_sz type"])]. [endsect] [section:release_notes_boost_1_66_00 Boost 1.66 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/41 GitHub Pull #41 (['"Data race in boost::interprocess::rbtree_best_fit"])]. [endsect] [section:release_notes_boost_1_65_00 Boost 1.65 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/37 GitHub Pull #37 (['"Conditionally replace deprecated/removed std::auto_ptr..."])]. [endsect] [section:release_notes_boost_1_64_00 Boost 1.64 Release] * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/12617 Trac #12617 (['"clock_gettime not available on OS X 10.11"])]. * [@https://svn.boost.org/trac/boost/ticket/12744 Trac #12744 (['"winapi::set_timer_resolution inadvertently changes timer resolution (Windows)"])]. * [@https://github.com/boostorg/interprocess/pull/32 GitHub Pull #32 (['"Conform to std::pointer_traits requirements"])]. * [@https://github.com/boostorg/interprocess/pull/33 GitHub Pull #33 (['"explicit cast to derived class" and "64/32 bit processes sharing"])]. * [@https://github.com/boostorg/interprocess/pull/34 GitHub Pull #34 (['"Update example to use multi_index::member instead of BOOST_MULTI_INDEX_MEMBER"])]. * [@https://github.com/boostorg/interprocess/pull/35 GitHub Pull #35 (['"Fixed options for cross-compilation"])]. * New experimental option `BOOST_INTERPROCESS_BOOTSTAMP_IS_SESSION_MANAGER_BASED` from Windows systems. This option derives the unique bootstamp used to name the folder where shared memory is placed from registry values associated with the session manager. This option only works on Vista and later systems and might be more stable than the default version. [endsect] [section:release_notes_boost_1_63_00 Boost 1.63 Release] * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/12499 Trac #12499 (['"Memory allocation fails"])]. * [@https://github.com/boostorg/interprocess/pull/30 GitHub Pull #30 (['"Provide extension point so library user can provide default temp folder"])]. * [@https://github.com/boostorg/interprocess/pull/31 GitHub Pull #31 (['"Add xsi_key(key_t) constructor"])]. [endsect] [section:release_notes_boost_1_62_00 Boost 1.62 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/27 GitHub Pull #27 (['"Fix undefined behavior"])]. [endsect] [section:release_notes_boost_1_61_00 Boost 1.61 Release] * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/23 GitHub Pull #23 (['"Fixed case sensitive for linux mingw"])]. [endsect] [section:release_notes_boost_1_60_00 Boost 1.60 Release] * Improved [classref boost::interprocess::offset_ptr offset_ptr] performance and removed any undefined behaviour. No special cases needed for different compilers. * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/11699 Trac #11699 (['"Forward declarations of std templates causes stack corruption under Visual Studio 2015"])]. [endsect] [section:release_notes_boost_1_59_00 Boost 1.59 Release] * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/5139 Trac #5139 (['"Initial Stream Position in Boost.Interprocess.Vectorstream"])]. * [@https://github.com/boostorg/interprocess/pull/19 GitHub Pull #19 (['"Fix exception visibility"])]. Thanks to Romain-Geissler. [endsect] [section:release_notes_boost_1_58_00 Boost 1.58 Release] * Reduced some compile-time dependencies. Updated to Boost.Container changes. * Fixed bugs: * [@https://github.com/boostorg/interprocess/pull/13 GitHub Pull #13 (['"haiku: we don't have XSI shared memory, so don't try to use it"])]. Thanks to Jessica Hamilton. [endsect] [section:release_notes_boost_1_57_00 Boost 1.57 Release] * Removed `unique_ptr`, now forwards boost::interprocess::unique_ptr to the general purpose `boost::movelib::unique_ptr` class from [*Boost.Move]. This implementation is closer to the standard `std::unique_ptr` implementation and it's better maintained. * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/10262 Trac #10262 (['"AIX 6.1 bug with variable definition hz"])]. * [@https://svn.boost.org/trac/boost/ticket/10229 Trac #10229 (['"Compiling errors in interprocess\detail\os_file_functions.hpp"])]. * [@https://svn.boost.org/trac/boost/ticket/10506 Trac #10506 (['"Infinite loop in create_or_open_file"])]. * [@https://github.com/boostorg/interprocess/pull/11 GitHub Pull #11 (['"Compile fix for BOOST_USE_WINDOWS_H"])]. * Reorganized Doxygen marks to obtain a better header reference. [endsect] [section:release_notes_boost_1_56_00 Boost 1.56 Release] * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/9221 Trac #9221 (['"message_queue deadlock on linux"])]. * [@https://svn.boost.org/trac/boost/ticket/9226 Trac #9226 (['"On some computers, Common Appdata is empty in registry, so boost interprocess cannot work"])]. * [@https://svn.boost.org/trac/boost/ticket/9262 Trac #9262 (['"windows_intermodule_singleton breaks when calling a Debug dll from a Release executable"])]. * [@https://svn.boost.org/trac/boost/ticket/9284 Trac #9284 (['"WaitForSingleObject(mutex) must handle WAIT_ABANDONED"])]. * [@https://svn.boost.org/trac/boost/ticket/9285 Trac #9285 (['"CreateMutex returns NULL if fails"])]. * [@https://svn.boost.org/trac/boost/ticket/9288 Trac #9288 (['"timed_wait does not check if it has expired"])]. * [@https://svn.boost.org/trac/boost/ticket/9408 Trac #9408 (['"Android does not support XSI_SHARED_MEMORY_OBJECTS"]]). * [@https://svn.boost.org/trac/boost/ticket/9729 Trac #9729 (['"crash on managed_external_buffer object construction"]]). * [@https://svn.boost.org/trac/boost/ticket/9767 Trac #9767 (['"bootstamp generation causes error in case of corrupt Windows Event Log"])]. * [@https://svn.boost.org/trac/boost/ticket/9835 Trac #9835 (['"Boost Interprocess fails to compile with Android NDK GCC 4.8, -Werror=unused-variable"])]. * [@https://svn.boost.org/trac/boost/ticket/9911 Trac #9911 (['"get_tmp_base_dir(...) failure"])]. * [@https://svn.boost.org/trac/boost/ticket/9946 Trac #9946 (['"ret_ptr uninitialized in init_atomic_func, fini_atomic_func"])]. * [@https://svn.boost.org/trac/boost/ticket/10011 Trac #10011 (['"segment_manager::find( unique_instance_t* ) fails to compile"])]. * [@https://svn.boost.org/trac/boost/ticket/10021 Trac #10021 (['"Interprocess and BOOST_USE_WINDOWS_H"])]. * [@https://svn.boost.org/trac/boost/ticket/10230 Trac #10230 (['"No Sleep in interprocess::winapi"])]. * [@https://github.com/boostorg/interprocess/pull/2 GitHub Pull #2] (['"Provide support for the Cray C++ compiler. The Cray compiler defines __GNUC__"]]). * [@https://github.com/boostorg/interprocess/pull/3 GitHub Pull #3] (['"Fix/mingw interprocess_exception throw in file_wrapper::priv_open_or_create"]]). * [*ABI breaking]: [@https://svn.boost.org/trac/boost/ticket/9221 #9221] showed that `BOOST_INTERPROCESS_MSG_QUEUE_CIRCULAR_INDEX` option of message queue, was completely broken so an ABI break was necessary to have a working implementation. * Simplified, refactored and unified (timed_)lock code based on try_lock(). There were several bugs when handling timeout expirations. * Changed the implementation of condition variables' destructors to allow POSIX semantics (the condition variable can be destroyed after all waiting threads have been woken up).. * Added `BOOST_INTERPROCESS_SHARED_DIR_PATH` option to define the shared directory used to place shared memory objects when implemented as memory mapped files. * Added support for `BOOST_USE_WINDOWS_H`. When this macro is defined Interprocess does not declare used Windows API function and types, includes all needed windows SDK headers and uses types and functions declared by the Windows SDK. * Added `get_size` to [classref ::boost:interprocess:windows_shared_memory]. [endsect] [section:release_notes_boost_1_55_00 Boost 1.55 Release] * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/7156 #7156] (['"interprocess buffer streams leak memory on construction"]]). * [@https://svn.boost.org/trac/boost/ticket/7164 #7164] (['"Two potential bugs with b::int::vector of b::i::weak_ptr"]]). * [@https://svn.boost.org/trac/boost/ticket/7860 #7860] (['"smart_ptr's yield_k and spinlock utilities can improve spinlock-based sychronization primitives"]]). * [@https://svn.boost.org/trac/boost/ticket/8277 #8277] (['"docs for named_mutex erroneously refer to interprocess_mutex"]]). * [@https://svn.boost.org/trac/boost/ticket/8976 #8976] (['"shared_ptr fails to compile if used with a scoped_allocator"]]). * [@https://svn.boost.org/trac/boost/ticket/9008 #9008] (['"conditions variables fast enough only when opening a multiprocess browser"]]). * [@https://svn.boost.org/trac/boost/ticket/9065 #9065] (['"atomic_cas32 inline assembly wrong on ppc32"]]). * [@https://svn.boost.org/trac/boost/ticket/9073 #9073] (['"Conflict names 'realloc'"]]). [endsect] [section:release_notes_boost_1_54_00 Boost 1.54 Release] * Added support for platform-specific flags to mapped_region (ticket #8030) * Fixed bugs: * [@https://svn.boost.org/trac/boost/ticket/7484 #7484], * [@https://svn.boost.org/trac/boost/ticket/7598 #7598], * [@https://svn.boost.org/trac/boost/ticket/7682 #7682], * [@https://svn.boost.org/trac/boost/ticket/7923 #7923], * [@https://svn.boost.org/trac/boost/ticket/7924 #7924], * [@https://svn.boost.org/trac/boost/ticket/7928 #7928], * [@https://svn.boost.org/trac/boost/ticket/7936 #7936], * [@https://svn.boost.org/trac/boost/ticket/8521 #8521], * [@https://svn.boost.org/trac/boost/ticket/8595 #8595]. * [*ABI breaking]: Changed bootstamp function in Windows to use EventLog service start time as system bootup time. Previously used `LastBootupTime` from WMI was unstable with time synchronization and hibernation and unusable in practice. If you really need to obtain pre Boost 1.54 behaviour define `BOOST_INTERPROCESS_BOOTSTAMP_IS_LASTBOOTUPTIME` from command line or `detail/workaround.hpp`. [endsect] [section:release_notes_boost_1_53_00 Boost 1.53 Release] * Fixed GCC -Wshadow warnings. * Experimental multiple allocation interface improved and changed again. Still unstable. * Replaced deprecated BOOST_NO_XXXX with newer BOOST_NO_CXX11_XXX macros. * [*ABI breaking]: changed node pool allocators internals for improved efficiency. * Fixed bug [@https://svn.boost.org/trac/boost/ticket/7795 #7795]. [endsect] [section:release_notes_boost_1_52_00 Boost 1.52 Release] * Added `shrink_by` and `advise` functions in `mapped_region`. * [*ABI breaking:] Reimplemented `message_queue` with a circular buffer index (the old behavior used an ordered array, leading to excessive copies). This should greatly increase performance but breaks ABI. Old behaviour/ABI can be used undefining macro `BOOST_INTERPROCESS_MSG_QUEUE_CIRCULAR_INDEX` in `boost/interprocess/detail/workaround.hpp` * Improved `message_queue` insertion time avoiding priority search for common cases (both array and circular buffer configurations). * Implemented `interproces_sharable_mutex` and `interproces_condition_any`. * Improved `offset_ptr` performance. * Added integer overflow checks. [endsect] [section:release_notes_boost_1_51_00 Boost 1.51 Release] * Synchronous and asynchronous flushing for `mapped_region::flush`. * [*Source & ABI breaking]: Removed `get_offset` method from `mapped_region` as it has no practical utility and `m_offset` member was not for anything else. * [*Source & ABI breaking]: Removed `flush` from `managed_shared_memory`. as it is unspecified according to POSIX: [@http://pubs.opengroup.org/onlinepubs/009695399/functions/msync.html ['"The effect of msync() on a shared memory object or a typed memory object is unspecified"] ]. * Fixed bug [@https://svn.boost.org/trac/boost/ticket/7152 #7152], [endsect] [section:release_notes_boost_1_50_00 Boost 1.50 Release] * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/3750 #3750], [@https://svn.boost.org/trac/boost/ticket/6727 #6727], [@https://svn.boost.org/trac/boost/ticket/6648 #6648], * Shared memory in windows has again kernel persistence: kernel bootstamp and WMI has received some fixes and optimizations. This causes incompatibility with Boost 1.48 and 1.49 but the user can comment `#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME` in the windows configuration part to get Boost 1.48 & Boost 1.49 behaviour. [endsect] [section:release_notes_boost_1_49_00 Boost 1.49 Release] * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/6531 #6531], [@https://svn.boost.org/trac/boost/ticket/6412 #6412], [@https://svn.boost.org/trac/boost/ticket/6398 #6398], [@https://svn.boost.org/trac/boost/ticket/6340 #6340], [@https://svn.boost.org/trac/boost/ticket/6319 #6319], [@https://svn.boost.org/trac/boost/ticket/6287 #6287], [@https://svn.boost.org/trac/boost/ticket/6265 #6265], [@https://svn.boost.org/trac/boost/ticket/6233 #6233], [@https://svn.boost.org/trac/boost/ticket/6147 #6147], [@https://svn.boost.org/trac/boost/ticket/6134 #6134], [@https://svn.boost.org/trac/boost/ticket/6058 #6058], [@https://svn.boost.org/trac/boost/ticket/6054 #6054], [@https://svn.boost.org/trac/boost/ticket/5772 #5772], [@https://svn.boost.org/trac/boost/ticket/5738 #5738], [@https://svn.boost.org/trac/boost/ticket/5622 #5622], [@https://svn.boost.org/trac/boost/ticket/5552 #5552], [@https://svn.boost.org/trac/boost/ticket/5518 #5518], [@https://svn.boost.org/trac/boost/ticket/4655 #4655], [@https://svn.boost.org/trac/boost/ticket/4452 #4452], [@https://svn.boost.org/trac/boost/ticket/4383 #4383], [@https://svn.boost.org/trac/boost/ticket/4297 #4297]. * Fixed timed functions in mutex implementations to fulfill POSIX requirements: ['Under no circumstance shall the function fail with a timeout if the mutex can be locked immediately. The validity of the abs_timeout parameter need not be checked if the mutex can be locked immediately.] [endsect] [section:release_notes_boost_1_48_00 Boost 1.48 Release] * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/2796 #2796], [@https://svn.boost.org/trac/boost/ticket/4031 #4031], [@https://svn.boost.org/trac/boost/ticket/4251 #4251], [@https://svn.boost.org/trac/boost/ticket/4452 #4452], [@https://svn.boost.org/trac/boost/ticket/4895 #4895], [@https://svn.boost.org/trac/boost/ticket/5077 #5077], [@https://svn.boost.org/trac/boost/ticket/5120 #5120], [@https://svn.boost.org/trac/boost/ticket/5123 #5123], [@https://svn.boost.org/trac/boost/ticket/5230 #5230], [@https://svn.boost.org/trac/boost/ticket/5197 #5197], [@https://svn.boost.org/trac/boost/ticket/5287 #5287], [@https://svn.boost.org/trac/boost/ticket/5294 #5294], [@https://svn.boost.org/trac/boost/ticket/5306 #5306], [@https://svn.boost.org/trac/boost/ticket/5308 #5308], [@https://svn.boost.org/trac/boost/ticket/5392 #5392], [@https://svn.boost.org/trac/boost/ticket/5409 #5409], * Added support to customize offset_ptr and allow creating custom managed segments that might be shared between 32 and 64 bit processes. * Shared memory in windows has again filesystem lifetime: kernel bootstamp and WMI use to get a reliable timestamp was causing a lot of trouble. [endsect] [section:release_notes_boost_1_46_00 Boost 1.46 Release] * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/4557 #4557], [@https://svn.boost.org/trac/boost/ticket/4979 #4979], [@https://svn.boost.org/trac/boost/ticket/4907 #4907], [@https://svn.boost.org/trac/boost/ticket/4895 #4895]. [endsect] [section:release_notes_boost_1_45_00 Boost 1.45 Release] * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/1080 #1080], [@https://svn.boost.org/trac/boost/ticket/3284 #3284], [@https://svn.boost.org/trac/boost/ticket/3439 #3439], [@https://svn.boost.org/trac/boost/ticket/3448 #3448], [@https://svn.boost.org/trac/boost/ticket/3582 #3582], [@https://svn.boost.org/trac/boost/ticket/3682 #3682], [@https://svn.boost.org/trac/boost/ticket/3829 #3829], [@https://svn.boost.org/trac/boost/ticket/3846 #3846], [@https://svn.boost.org/trac/boost/ticket/3914 #3914], [@https://svn.boost.org/trac/boost/ticket/3947 #3947], [@https://svn.boost.org/trac/boost/ticket/3950 #3950], [@https://svn.boost.org/trac/boost/ticket/3951 #3951], [@https://svn.boost.org/trac/boost/ticket/3985 #3985], [@https://svn.boost.org/trac/boost/ticket/4010 #4010], [@https://svn.boost.org/trac/boost/ticket/4417 #4417], [@https://svn.boost.org/trac/boost/ticket/4019 #4019], [@https://svn.boost.org/trac/boost/ticket/4039 #4039], [@https://svn.boost.org/trac/boost/ticket/4218 #4218], [@https://svn.boost.org/trac/boost/ticket/4230 #4230], [@https://svn.boost.org/trac/boost/ticket/4250 #4250], [@https://svn.boost.org/trac/boost/ticket/4297 #4297], [@https://svn.boost.org/trac/boost/ticket/4350 #4350], [@https://svn.boost.org/trac/boost/ticket/4352 #4352], [@https://svn.boost.org/trac/boost/ticket/4426 #4426], [@https://svn.boost.org/trac/boost/ticket/4516 #4516], [@https://svn.boost.org/trac/boost/ticket/4524 #4524], [@https://svn.boost.org/trac/boost/ticket/4557 #4557], [@https://svn.boost.org/trac/boost/ticket/4606 #4606], [@https://svn.boost.org/trac/boost/ticket/4685 #4685], [@https://svn.boost.org/trac/boost/ticket/4694 #4694]. * Added support for standard rvalue reference move semantics (tested on GCC 4.5 and VC10). * Permissions can be detailed for interprocess named resources. * `mapped_region::flush` initiates disk flushing but does not guarantee it's completed when returns, since it is not portable. * FreeBSD and MacOS now use posix semaphores to implement named semaphores and mutex. [endsect] [section:release_notes_boost_1_41_00 Boost 1.41 Release] * Support for POSIX shared memory in Mac OS. * [*ABI breaking]: Generic `semaphore` and `named_semaphore` now implemented more efficiently with atomic operations. * More robust file opening in Windows platforms with active Anti-virus software. [endsect] [section:release_notes_boost_1_40_00 Boost 1.40 Release] * Windows shared memory is created in Shared Documents folder so that it can be shared between services and processes * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/2967 #2967], [@https://svn.boost.org/trac/boost/ticket/2973 #2973], [@https://svn.boost.org/trac/boost/ticket/2992 #2992], [@https://svn.boost.org/trac/boost/ticket/3138 #3138], [@https://svn.boost.org/trac/boost/ticket/3166 #3166], [@https://svn.boost.org/trac/boost/ticket/3205 #3205]. [endsect] [section:release_notes_boost_1_39_00 Boost 1.39 Release] * Added experimental `stable_vector` container. * `shared_memory_object::remove` has now POSIX `unlink` semantics and `file_mapping::remove` was added to obtain POSIX `unlink` semantics with mapped files. * Shared memory in windows has now kernel lifetime instead of filesystem lifetime: shared memory will disappear when the system reboots. * Updated move semantics. * Fixed bugs [@https://svn.boost.org/trac/boost/ticket/2722 #2722], [@https://svn.boost.org/trac/boost/ticket/2729 #2729], [@https://svn.boost.org/trac/boost/ticket/2766 #2766], [@https://svn.boost.org/trac/boost/ticket/1390 #1390], [@https://svn.boost.org/trac/boost/ticket/2589 #2589], [endsect] [section:release_notes_boost_1_38_00 Boost 1.38 Release] * Updated documentation to show rvalue-references funcions instead of emulation functions. * More non-copyable classes are now movable. * Move-constructor and assignments now leave moved object in default-constructed state instead of just swapping contents. * Several bugfixes ( [@https://svn.boost.org/trac/boost/ticket/2391 #2391], [@https://svn.boost.org/trac/boost/ticket/2431 #2431], [@https://svn.boost.org/trac/boost/ticket/1390 #1390], [@https://svn.boost.org/trac/boost/ticket/2570 #2570], [@https://svn.boost.org/trac/boost/ticket/2528 #2528]. [endsect] [section:release_notes_boost_1_37_00 Boost 1.37 Release] * Containers can be used now in recursive types. * Added `BOOST_INTERPROCESS_FORCE_GENERIC_EMULATION` macro option to force the use of generic emulation code for process-shared synchronization primitives instead of native POSIX functions. * Added placement insertion members to containers * `boost::posix_time::pos_inf` value is now handled portably for timed functions. * Update some function parameters from `iterator` to `const_iterator` in containers to keep up with the draft of the next standard. * Documentation fixes. [endsect] [section:release_notes_boost_1_36_00 Boost 1.36 Release] * Added anonymous shared memory for UNIX systems. * Fixed erroneous `void` return types from `flat_map::erase()` functions. * Fixed missing move semantics on managed memory classes. * Added copy_on_write and open_read_only options for shared memory and mapped file managed classes. * [*ABI breaking]: Added to `mapped_region` the mode used to create it. * Corrected instantiation errors in void allocators. * `shared_ptr` is movable and supports aliasing. [endsect] [section:release_notes_boost_1_35_00 Boost 1.35 Release] * Added auxiliary utilities to ease the definition and construction of [classref boost::interprocess::shared_ptr shared_ptr], [classref boost::interprocess::weak_ptr weak_ptr] and `unique_ptr`. Added explanations and examples of these smart pointers in the documentation. * Optimized vector: * 1) Now works with raw pointers as much as possible when using allocators defining `pointer` as an smart pointer. This increases performance and improves compilation times. * 2) A bit of metaprogramming to avoid using move_iterator when the type has trivial copy constructor or assignment and improve performance. * 3) Changed custom algorithms with standard ones to take advantage of optimized standard algorithms. * 4) Removed unused code. * [*ABI breaking]: Containers don't derive from allocators, to avoid problems with allocators that might define virtual functions with the same names as container member functions. That would convert container functions in virtual functions and might disallow some of them if the returned type does not lead to a covariant return. Allocators are now stored as base classes of internal structs. * Implemented [classref boost::interprocess::named_mutex named_mutex] and [classref boost::interprocess::named_semaphore named_semaphore] with POSIX named semaphores in systems supporting that option. [classref boost::interprocess::named_condition named_condition] has been accordingly changed to support interoperability with [classref boost::interprocess::named_mutex named_mutex]. * Reduced template bloat for node and adaptive allocators extracting node implementation to a class that only depends on the memory algorithm, instead of the segment manager + node size + node number... * Fixed bug in `mapped_region` in UNIX when mapping address was provided but the region was mapped in another address. * Added `aligned_allocate` and `allocate_many` functions to managed memory segments. * Improved documentation about managed memory segments. * [*Boost.Interprocess] containers are now documented in the Reference section. * Correction of typos and documentation errors. * Added `get_instance_name`, `get_instance_length` and `get_instance_type` functions to managed memory segments. * Corrected suboptimal buffer expansion bug in `rbtree_best_fit`. * Added iteration of named and unique objects in a segment manager. * Fixed leak in [classref boost::interprocess::vector vector]. * Added support for Solaris. * Optimized [classref boost::interprocess::segment_manager segment_manager] to avoid code bloat associated with templated instantiations. * Fixed bug for UNIX: No slash ('/') was being added as the first character for shared memory names, leading to errors in some UNIX systems. * Fixed bug in VC-8.0: Broken function inlining in core offset_ptr functions. * Code examples changed to use new BoostBook code import features. * Added aligned memory allocation function to memory algorithms. * Fixed bug in `deque::clear()` and `deque::erase()`, they were declared private. * Fixed bug in `deque::erase()`. Thanks to Steve LoBasso. * Fixed bug in `atomic_dec32()`. Thanks to Glenn Schrader. * Improved (multi)map/(multi)set constructors taking iterators. Now those have linear time if the iterator range is already sorted. * [*ABI breaking]: (multi)map/(multi)set now reduce their node size. The color bit is embedded in the parent pointer. Now, the size of a node is the size of 3 pointers in most systems. This optimization is activated for raw and `offset_ptr` pointers. * (multi)map/(multi)set now reuse memory from old nodes in the assignment operator. * [*ABI breaking]: Implemented node-containers based on intrusive containers. This saves code size, since many instantiations share the same algorithms. * Corrected code to be compilable with Visual C++ 8.0. * Added function to zero free memory in memory algorithms and the segment manager. This function is useful for security reasons and to improve compression ratios for files created with `managed_mapped_file`. * Added support for intrusive index types in managed memory segments. Intrusive indexes save extra memory allocations to allocate the index since with just one allocation, we allocate room for the value, the name and the hook to insert the object in the index. * Created new index type: [*iset_index]. It's an index based on an intrusive set (rb-tree). * Created new index type: [*iunordered_set_index]. It's an index based on a pseudo-intrusive unordered set (hash table). * [*ABI breaking]: The intrusive index [*iset_index] is now the default index type. * Optimized vector to take advantage of `boost::has_trivial_destructor`. This optimization avoids calling destructors of elements that have a trivial destructor. * Optimized vector to take advantage of `has_trivial_destructor_after_move` trait. This optimization avoids calling destructors of elements that have a trivial destructor if the element has been moved (which is the case of many movable types). This trick was provided by Howard Hinnant. * Added security check to avoid integer overflow bug in allocators and named construction functions. * Added alignment checks to forward and backwards expansion functions. * Fixed bug in atomic functions for PPC. * Fixed race-condition error when creating and opening a managed segment. * Added adaptive pools. * [*Source breaking]: Changed node allocators' template parameter order to make them easier to use. * Added support for native windows shared memory. * Added more tests. * Corrected the presence of private functions in the reference section. * Added function (`deallocate_free_chunks()`) to manually deallocate completely free chunks from node allocators. * Implemented N1780 proposal to LWG issue 233: ['Insertion hints in associative containers] in interprocess [classref boost::interprocess::multiset multiset] and [classref boost::interprocess::multimap multimap] classes. * [*Source breaking]: A shared memory object is now used including `shared_memory_object.hpp` header instead of `shared memory.hpp`. * [*ABI breaking]: Changed global mutex when initializing managed shared memory and memory mapped files. This change tries to minimize deadlocks. * [*Source breaking]: Changed shared memory, memory mapped files and mapped region's open mode to a single `mode_t` type. * Added extra WIN32_LEAN_AND_MEAN before including DateTime headers to avoid socket redefinition errors when using Interprocess and Asio in windows. * [*ABI breaking]: `mapped_region` constructor no longer requires classes derived from memory_mappable, but classes must fulfill the MemoryMappable concept. * Added in-place reallocation capabilities to basic_string. * [*ABI breaking]: Reimplemented and optimized small string optimization. The narrow string class has zero byte overhead with an internal 11 byte buffer in 32 systems! * Added move semantics to containers. Improves performance when using containers of containers. * [*ABI breaking]: End nodes of node containers (list, slist, map/set) are now embedded in the containers instead of allocated using the allocator. This allows no-throw move-constructors and improves performance. * [*ABI breaking]: [*slist] and [*list] containers now have constant-time ['size()] function. The size of the container is added as a member. [endsect] [endsect] [section:books_and_links Books and interesting links] Some useful references about the C++ programming language, C++ internals, shared memory, allocators and containers used to design [*Boost.Interprocess]. [section:references_books Books] * Great book about multithreading, and POSIX: [*['"Programming with Posix Threads"]], [*David R. Butenhof] * The UNIX inter-process bible: [*['"UNIX Network Programming, Volume 2: Interprocess Communications"]], [*W. Richard Stevens] * Current STL allocator issues: [*['"Effective STL"]], [*Scott Meyers] * My C++ bible: [*['"Thinking in C++, Volume 1 & 2"]], [*Bruce Eckel and Chuck Allison] * The book every C++ programmer should read: [*['"Inside the C++ Object Model"]], [*Stanley B. Lippman] * A must-read: [*['"ISO/IEC TR 18015: Technical Report on C++ Performance"]], [*ISO WG21-SC22 members.] [endsect] [section:references_links Links] * A framework to put the STL in shared memory: [@http://allocator.sourceforge.net/ ['"A C++ Standard Allocator for the Standard Template Library"] ]. * Instantiating C++ objects in shared memory: [@http://www.cs.ubc.ca/local/reading/proceedings/cascon94/htm/english/abs/hon.htm ['"Using objects in shared memory for C++ application"] ]. * A shared memory allocator and relative pointer: [@http://home.earthlink.net/~joshwalker1/writing/SharedMemory.html ['"Taming Shared Memory"] ]. [endsect] [endsect] [section:future_improvements Future improvements...] There are some Interprocess features that I would like to implement and some [*Boost.Interprocess] code that can be much better. Let's see some ideas: [section:win32_sync Win32 synchronization is too basic] Win32 version of shared mutexes and shared conditions are based on "spin and wait" atomic instructions. This leads to poor performance and does not manage any issues like priority inversions. We would need very serious help from threading experts on this. And I'm not sure that this can be achieved in user-level software. Posix based implementations use PTHREAD_PROCESS_SHARED attribute to place mutexes in shared memory, so there are no such problems. I'm not aware of any implementation that simulates PTHREAD_PROCESS_SHARED attribute for Win32. We should be able to construct these primitives in memory mapped files, so that we can get filesystem persistence just like with POSIX primitives. [endsect] [section:future_objectnames Use of wide character names on Boost.Interprocess basic resources] Currently Interprocess only allows *char* based names for basic named objects. However, several operating systems use *wchar_t* names for resources (mapped files, for example). In the future Interprocess should try to present a portable narrow/wide char interface. To do this, it would be useful to have a boost wstring <-> string conversion utilities to translate resource names (escaping needed characters that can conflict with OS names) in a portable way. It would be interesting also the use of [*boost::filesystem] paths to avoid operating system specific issues. [endsect] [section:future_security Security attributes] [*Boost.Interprocess] does not define security attributes for shared memory and synchronization objects. Standard C++ also ignores security attributes with files so adding security attributes would require some serious work. [endsect] [section:future_ipc Future inter-process communications] [*Boost.Interprocess] offers a process-shared message queue based on [*Boost.Interprocess] primitives like mutexes and conditions. I would want to develop more mechanisms, like stream-oriented named fifo so that we can use it with a iostream-interface wrapper (we can imitate Unix pipes). C++ needs more complex mechanisms and it would be nice to have a stream and datagram oriented PF_UNIX-like mechanism in C++. And for very fast inter-process remote calls Solaris doors is an interesting alternative to implement for C++. But the work to implement PF_UNIX-like sockets and doors would be huge (and it might be difficult in a user-level library). Any network expert volunteer? [endsect] [endsect] [endsect] [section:indexes_reference Indexes and Reference] [section:index Indexes] [include auto_index_helpers.qbk] [named_index class_name Class Index] [named_index typedef_name Typedef Index] [named_index function_name Function Index] [/named_index macro_name Macro Index] [/index] [endsect] [xinclude autodoc.xml] [endsect]