Skip to content
hgryoo · Docs

CUBRID Base / Infrastructure — Section Overview

Contents:

What this section covers

Section titled “What this section covers”

The base / infrastructure section groups the CUBRID source modules that live under src/base/ and that every layer above — storage engine, query processor, transaction and recovery, PL family — composes with. They are not part of any one of those layers; they are the substrate the layers sit on.

Two families:

  1. Custom memory allocators. Sit between the OS malloc and the engine’s hot-path subsystems, decoupling per-thread allocation from a global free-list and recycling fixed-size objects out of slab pools.
  2. Lock-free primitives. The concurrent data structures that coordinate the page buffer, the lock manager, the MVCC table, vacuum, and CDC — anchored on a single transactional reclamation spine that lets retired nodes be safely freed without locks.

Both families are infrastructure in the sense that no detail doc in another subcategory could be self-contained without referring back to them. The allocator family backs the C and C++ heaps that every subsystem uses; the lock-free family backs every concurrent table the engine reads under load. Pulling them out of subcategory: storage-engine (where AREA historically lived) and out of an unsubcategorised limbo (where the lockfree docs lived) into a single subcategory: base-infra matches the source tree’s src/base/ directory and gives the table-of-contents a place to anchor cross-cutting primitives.

The substrate

Section titled “The substrate”
flowchart TB
  subgraph CONSUMERS["Consumers (every other section)"]
    STORAGE["storage-engine<br/>page-buffer, heap, btree, ehash, ..."]
    QP["query-processing<br/>parser, optimizer, executor, list-file, ..."]
    TX["txn-recovery<br/>mvcc, lock-manager, log-manager, vacuum, ..."]
    PL["pl-language<br/>cub_pl, JavaSP, PL/CSQL"]
  end

  subgraph LFREE["Lock-free primitives"]
    LFOVER["cubrid-lockfree-overview.md<br/>(map + reclamation spine)"]
    LFTX["cubrid-lockfree-transaction.md"]
    LFHM["cubrid-lockfree-hashmap.md"]
    LFFL["cubrid-lockfree-freelist.md"]
    LFCQ["cubrid-lockfree-circular-queue.md"]
    LFBM["cubrid-lockfree-bitmap.md"]
  end

  subgraph ALLOC["Custom allocators"]
    AREA["cubrid-common-area.md<br/>(AREA slab pool)"]
    PRIV["cubrid-private-allocator.md<br/>(per-thread Lea heap +<br/>C++ STL wrapper)"]
  end

  subgraph OS["OS layer"]
    MALLOC["malloc / free"]
  end

  CONSUMERS --> LFREE
  CONSUMERS --> ALLOC
  AREA --> LFBM
  AREA --> PRIV
  PRIV --> MALLOC
  LFREE -.-> PRIV

  classDef section fill:#eef,stroke:#88a
  class LFREE,ALLOC section

Three properties are worth pointing at:

  • AREA depends on the private allocator and on the lockfree bitmap. AREA’s per-block free-cell tracker is a lockfree::bitmap (so the choice to put both in the same section is partly a documentation convenience and partly a reflection of the actual coupling). AREA’s block arrays are allocated through the private allocator on SERVER_MODE.
  • The lockfree primitives are independent of the allocators. They allocate their own nodes — most through the freelist’s back-buffer block allocator (cubrid-lockfree-freelist.md), some directly through new / db_private_alloc. The reclamation spine (cubrid-lockfree-transaction.md) is what unifies them.
  • The OS layer is malloc/free, not anything more sophisticated. CUBRID does not bundle tcmalloc, jemalloc, or mimalloc; the per-thread arena story is carried by Doug Lea’s dlmalloc (vendored under customheaps), instantiated once per worker thread.

Reading order

Section titled “Reading order”
  1. cubrid-private-allocator.md — per-thread Lea heap, the cubmem::private_allocator<T> STL wrapper, build-mode routing (SERVER / SA / CS). Read this first; everything else allocates against it directly or transitively.
  2. cubrid-common-area.md — slab pool for fixed-size objects (DB_VALUE, TP_DOMAIN, …). Read after the private allocator since it uses the private heap as backing store and the lockfree bitmap as its per-block free tracker.
  3. cubrid-lockfree-overview.md — map of the lock-free primitives and the reclamation spine. Read this before any per-primitive doc.
  4. cubrid-lockfree-transaction.md — the reclamation spine. Read second; the bitmap, freelist, hashmap, and circular queue all sit on top of it.
  5. Then any of the per-primitive lockfree docs (cubrid-lockfree-bitmap.md, cubrid-lockfree-freelist.md, cubrid-lockfree-hashmap.md, cubrid-lockfree-circular-queue.md) as the topic demands.

Detail-doc summaries

Section titled “Detail-doc summaries”
DocSummary
cubrid-private-allocator.mdPer-thread Lea-heap arena (Doug Lea’s dlmalloc vendored under customheaps) instantiated once per THREAD_ENTRY, fronted by db_private_alloc / _free / _realloc macros that route SERVER_MODE allocations to the thread’s heap, CS_MODE to the workspace, and SA_MODE through a PRIVATE_MALLOC_HEADER-tagged dispatch. C++ STL wrapper cubmem::private_allocator<T> lets STL containers participate; private_unique_ptr<T> and PRIVATE_BLOCK_ALLOCATOR are the convenience layers.
cubrid-common-area.mdSlab-style pool allocator — chained 256-block BLOCKSET arrays of fixed-cell blocks for same-size objects (DB_VALUE, TP_DOMAIN, OBJ_TEMPLATE, set objects, …), lock-free per-block bitmap, single hint pointer for the common case.
cubrid-lockfree-overview.mdMap of CUBRID’s lock-free primitives — the legacy C lock_free.{h,c} family and the modern C++ lockfree::* namespace — anchored on a single transactional reclamation spine that every other lock-free structure in the engine sits on.
cubrid-lockfree-transaction.mdSystem / table / descriptor / address-marker reclamation — per-data-structure transaction id, per-thread descriptors that bracket reads, periodic minimum-active-id scan that tells the freelist when a retired node is no longer reachable from any live reader.
cubrid-lockfree-bitmap.mdChunked atomic-word bitmap — std::atomic<unsigned int> chunks, two chunking styles (one-chunk full-usage, list-of-chunks usage-bounded), CAS bit-flip, round-robin start hint that bumps atomically per get_entry under SERVER_MODE.
cubrid-lockfree-circular-queue.mdBounded MPMC ring with two cursor atomics and a per-slot block-flag word — used for vacuum log-block dispatch, page-buffer victim handoff, and CDC log-info forwarding.
cubrid-lockfree-freelist.mdTyped freelist<T> with a single available stack, a one-block back-buffer that swaps in lazily so concurrent claimers do not race to allocate, an on_reclaim payload hook, and a clearly-documented ABA window in the pop path bounded by the back-buffer time.
cubrid-lockfree-hashmap.mdHarris–Michael chained hash with optional per-entry mutex, in two parallel implementations (legacy C lf_hash_* and modern C++ lockfree::hashmap<K,T>) bridged by cubthread::lockfree_hashmap<K,T> whose m_type ∈ {OLD, NEW} is decided at init by PRM_ID_ENABLE_NEW_LFHASH.

Adjacent sections

Section titled “Adjacent sections”
  • cubrid-overview-storage-engine.md — page buffer, heap, B+Tree, extendible hash, double-write buffer, LOB, TDE. Every storage layer allocates through the private allocator; the page buffer and the lock manager use the lockfree primitives. AREA used to live in this section; it has been moved here because it is a memory-allocator concern that any layer (not just storage) can use.
  • cubrid-overview-server-architecture.md — thread pool, connection pool, error management, system parameters, monitoring. The thread-pool entries (cubthread::entry) are what carry private_heap_id; error / parameters / monitoring are also src/base/ substrate but are kept in server-architecture because they are also touched by the network stack and the broker. If/when the boundary is re-drawn, they may migrate here.
  • cubrid-architecture-overview.md §“Cross-cutting infrastructure” — front-door router that names base / infra alongside the rest of the cross-cutting surface.
  • cubrid-design-philosophy.md — the historical reason CUBRID uses a vendored dlmalloc rather than tcmalloc / jemalloc.