When considering performance issues for file based access, it is important to minimise disk IO and disk head movement in particular.
At the lowest level the read-write store buffers write requests, and orders them to minimise disk head movement.
The buffer size is not large - ~20K - but provides significant advantages when writing many
small chunks of data. Typical GPO objects are often less than 128 bytes.
The store allocation algorithm uses fixed size allocators.
Specifically, allocators exist for 64, 128, 256, 512, 1024, 2048, 4096 and 8192 byte allocation blocks.
The store provides larger chunks of disk allocation to the allocators on request. These chunks are never smaller than 8K. So a 64byte allocator will allocate 256 bits worth at a time.
The address of each of these allocations is stored within the allocator, and since this address is always on an 8K boundary, the effective addressing range of a 32 bit stored address can be 16 terabytes.
The address reference returned from an allocation request, in fact is a number of index offsets into the allocation structures. First to the allocation block, and then to a computed bit-offet within the block. This is then used to determine the physical disk address.
This scheme not only increases the addressing range to 16 terabytes, but also allows for constant time access to the allocation block, when an update is required.
Allocation blocks with free storage available, are further stored on a sorted list of blocks, ensuring again that allocation performance is maintained as the store grows.
The store does not provide conventional "blob" allocation. Instead all allocation uses a stream protocol. When an 8K block is used, the last four bytes are reserved to indicate whether there is a continuation block.
In this way, "blob" allocation is achieved using the fixed allocators, this has a number of performance and scalability advantages, from re-use of blocks on re-allocation to supporting streaming of large data streams to the store, such as media files.
The file size is grown in stages, and in proportion to the current file size.
The storage allocators are stored in a meta allocation area at the end of the file. When the file is extended, only the storage allocators need to be moved. These are shifted to the new end of file.
The new store header contains information on the current file allocation parameters - where the allocators are etc..
Transaction scopes not only provide update atomicity they also improve performance.
This is because after each committed update the modified allocators must be written to disk, along with the new store header record and lastly the pointer to the "current" store header.
This is the commit overhead.
By ensuring that updates are grouped together in transactions not only do you
make sure that only if all is okay does the update succeed, but you also reduce the transaction overhead
to a final set of writes.
Furthermore, the sorted write buffering may mean that some writes are not required, when the relevant area is subsequently free'd before it is written to.
The main memory requirement for the Read-Write Store is for the fixed allocators.
Since they are used for address decoding as well as re-allocation, they must be present at all times. Each allocator requires a little over 1K of java memory, and saves itself to a 1K block in the meta allocation area in the file. Different allocator sizes achieve different bit densities - in other words a 64 byte allocator allocates 256 64 byte bits at a time, while a 2K allocator will only allocate 32 2k byte bits.
In short, this means that the worst case scenario is that only 512 bytes of the allocator are used indicate allocated storage. So 1K of allocator memory will control 512 * 8 allocations or 4K.
So a store with 10 million allocations will require approximately 2500 allocators, leading to a java VM resource requirement of ~2.5MB.