Class UpfrontAllocatingPageSource

java.lang.Object
org.terracotta.offheapstore.paging.UpfrontAllocatingPageSource
All Implemented Interfaces:
PageSource
public class UpfrontAllocatingPageSource extends Object implements PageSource
An upfront allocating direct byte buffer source.

This buffer source implementation allocates all of its required storage up-front in fixed size chunks. Runtime allocations are then satisfied using slices from these initial chunks.

  • Field Details

    • ALLOCATION_LOG_LOCATION

      public static final String ALLOCATION_LOG_LOCATION

    • LOGGER

      private static final org.slf4j.Logger LOGGER

    • PROGRESS_LOGGING_STEP_SIZE

      private static final double PROGRESS_LOGGING_STEP_SIZE
      See Also:

    • PROGRESS_LOGGING_THRESHOLD

      private static final long PROGRESS_LOGGING_THRESHOLD

    • REGION_COMPARATOR

      private static final Comparator<Page> REGION_COMPARATOR

    • risingThresholds

      private final SortedMap<Long,Runnable> risingThresholds

    • fallingThresholds

      private final SortedMap<Long,Runnable> fallingThresholds

    • sliceAllocators

      private final List<PowerOfTwoAllocator> sliceAllocators

    • victimAllocators

      private final List<PowerOfTwoAllocator> victimAllocators

    • buffers

      private final List<ByteBuffer> buffers

    • victims

      private final List<NavigableSet<Page>> victims

    • availableSet

      private volatile int availableSet

  • Constructor Details

    • UpfrontAllocatingPageSource

      public UpfrontAllocatingPageSource(BufferSource source, long toAllocate, int chunkSize)
      Create an up-front allocating buffer source of toAllocate total bytes, in chunkSize byte chunks.
      Parameters:
      source - source from which initial buffers will be allocated
      toAllocate - total space to allocate in bytes
      chunkSize - chunkSize size to allocate in bytes

    • UpfrontAllocatingPageSource

      public UpfrontAllocatingPageSource(BufferSource source, long toAllocate, int maxChunk, int minChunk)
      Create an up-front allocating buffer source of toAllocate total bytes, in maximally sized chunks, within the given bounds.
      Parameters:
      source - source from which initial buffers will be allocated
      toAllocate - total space to allocate in bytes
      maxChunk - the largest chunk size in bytes
      minChunk - the smallest chunk size in bytes

    • UpfrontAllocatingPageSource

      private UpfrontAllocatingPageSource(BufferSource source, long toAllocate, int maxChunk, int minChunk, boolean fixed)
      Create an up-front allocating buffer source of toAllocate total bytes, in maximally sized chunks, within the given bounds.

      By default we try to allocate chunks of maxChunk size. However, unless fixed is true, in case of allocation failure, we will try to allocate half-smaller chunks. We do not allocate chunks smaller than minChunk though.

      Parameters:
      source - source from which initial buffers will be allocated
      toAllocate - total space to allocate in bytes
      maxChunk - the largest chunk size in bytes
      minChunk - the smallest chunk size in bytes
      fixed - if the chunks should all be of size maxChunk or can be smaller

  • Method Details

    • getCapacity

      public long getCapacity()
      Return the total allocated capacity, used or not
      Returns:
      the total capacity

    • allocate

      public Page allocate(int size, boolean thief, boolean victim, OffHeapStorageArea owner)
      Allocates a byte buffer of at least the given size.

      This BufferSource is limited to allocating regions that are a power of two in size. Supplied sizes are therefore rounded up to the next largest power of two.

      Specified by:
      allocate in interface PageSource
      Parameters:
      size - size of page to allocate
      thief - true if the allocation can steal space from victims
      victim - true if the allocated page should be eligible for stealing
      owner - owner from which subsequent steal should occur
      Returns:
      a buffer of at least the given size

    • allocateAsThief

      private Page allocateAsThief(int size, boolean victim, OffHeapStorageArea owner)

    • findVictimPages

      private List<Page> findVictimPages(int chunk, int address, int size)

    • allocateFromFree

      private Page allocateFromFree(int size, boolean victim, OffHeapStorageArea owner)

    • free

      public void free(Page page)
      Frees the supplied buffer.

      If the given buffer was not allocated by this source or has already been freed then an AssertionError is thrown.

      Specified by:
      free in interface PageSource

    • getAllocatedSize

      public long getAllocatedSize()

    • getAllocatedSizeUnSync

      public long getAllocatedSizeUnSync()

    • isUnavailable

      private boolean isUnavailable(int size)

    • markAllAvailable

      private void markAllAvailable()

    • markUnavailable

      private void markUnavailable(int size)

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • fireThresholds

      private void fireThresholds(long incoming, long outgoing)

    • addAllocationThreshold

      public Runnable addAllocationThreshold(UpfrontAllocatingPageSource.ThresholdDirection direction, long threshold, Runnable action)
      Adds an allocation threshold action.

      There can be only a single action associated with each unique direction and threshold combination. If an action is already associated with the supplied combination then the action is replaced by the new action and the old action is returned.

      Actions are fired on passing through the supplied threshold and are called synchronously with the triggering allocation. This means care must be taken to avoid mutating any map that uses this page source from within the action otherwise deadlocks may result. Exceptions thrown by the action will be caught and logged by the page source and will not be propagated on the allocating thread.

      Parameters:
      direction - new actions direction
      threshold - new actions threshold level
      action - fired on breaching the threshold
      Returns:
      the replaced action or null if no action was present.

    • removeAllocationThreshold

      public Runnable removeAllocationThreshold(UpfrontAllocatingPageSource.ThresholdDirection direction, long threshold)
      Removes an allocation threshold action.

      Removes the allocation threshold action for the given level and direction.

      Parameters:
      direction - registered actions direction
      threshold - registered actions threshold level
      Returns:
      the removed condition or null if no action was present.

    • allocateBackingBuffers

      private static Collection<ByteBuffer> allocateBackingBuffers(BufferSource source, long toAllocate, int maxChunk, int minChunk, boolean fixed)
      Allocate multiple buffers to fulfill the requested memory toAllocate. We first divide toAllocate in chunks of size maxChunk and try to allocate them in parallel on all available processors. If one chunk fails to be allocated, we try to allocate two chunks of maxChunk / 2. If this allocation fails, we continue dividing until we reach of size of minChunk. If at that moment, the allocation still fails, an IllegalArgumentException is thrown.

      When fixed is requested, we will only allocated buffers of maxChunk size. If allocation fails, an IllegalArgumentException is thrown without any division.

      If the allocation is interrupted, the method will ignore it and continue allocation. It will then return with the interrupt flag is set.

      Parameters:
      source - source used to allocate memory buffers
      toAllocate - total amount of memory to allocate
      maxChunk - maximum size of a buffer. This is the targeted size for all buffers if everything goes well
      minChunk - minimum buffer size allowed
      fixed - if all buffers should have a the same size (except the last one with toAllocate % maxChunk != 0, if true, minChunk isn't used
      Returns:
      the list of allocated buffers
      Throws:
      IllegalArgumentException - when we fail to allocate the requested memory

    • bufferAllocation

      private static Collection<ByteBuffer> bufferAllocation(BufferSource source, int toAllocate, int minChunk, boolean fixed, PrintStream allocatorLog, long start)

    • uninterruptibleGet

      private static <T> T uninterruptibleGet(Future<T> future)

    • createAllocatorLog

      private static PrintStream createAllocatorLog(long max, int maxChunk, int minChunk)