Where this document fits: The high-level analysis
cubrid-double-write-buffer.md covers design intent and theoretical
background. This document traces every branch and field at the
code level. Each chapter is self-contained, but reading in order
follows the full lifecycle of a single flushed page through the DWB
— slot reservation, block fill, double write, and crash recovery.
This chapter answers: what in-memory and on-disk structures make up the DWB, and how do their fields and pointers relate? Every other chapter assumes the seven structs and one singleton defined here. Torn-page theory is in the high-level companion cubrid-double-write-buffer.md (“Theoretical Background”, “Torn-page protection”). All definitions live in src/storage/double_write_buffer.cpp except double_write_slot (in the .hpp — the only type a caller names); the instance is a file-scope static.
The DWB is a fixed array of blocks. Each block owns one contiguous write_buffer (the only real page memory) plus two parallel views: slots (one DWB_SLOT per page) and flush_volumes_info (one per distinct destination volume). The singleton dwb_Global owns the block array, a lock-free VPID hash map, and a 64-bit position_with_flags word encoding fill cursor + per-block state (Chapter 2).
flowchart TD
G["dwb_Global<br/>(double_write_buffer)"] --> B0["blocks[0]<br/>(double_write_block)"]
G --> B1["blocks[1] ... blocks[num_blocks-1]"]
G --> HM["slots_hashmap<br/>(lockfree_hashmap<VPID, dwb_slots_hash_entry>)"]
G --> PWF["position_with_flags (UINT64)"]
B0 --> WB["write_buffer<br/>(char* — the ONLY backing memory)"]
B0 --> SL["slots[]<br/>(DWB_SLOT — a VIEW onto write_buffer)"]
B0 --> FV["flush_volumes_info[]<br/>(FLUSH_VOLUME_INFO — one per dest volume)"]
SL -. "io_page == write_buffer + i*IO_PAGESIZE" .-> WB
HM --> HE["dwb_slots_hash_entry"]
HE -. "slot" .-> SL
A slot is a handle to one page staged in a block’s write_buffer — what a caller gets from dwb_set_data_on_next_slot/dwb_add_page and what the flush path iterates.
FILEIO_PAGE *io_page; /* <- NOT owned; points into block->write_buffer */
VPID vpid;
LOG_LSA lsa;
bool ensure_metadata;
unsignedint position_in_block;
unsignedint block_no;
};
Field
Role
Why it exists
io_page
Staged page image in the block’s write_buffer. Borrowed, never freed.
Flush writes page bytes with no second copy.
vpid
(volid, pageid) identity held now.
Hash key + home address; from io_page->prv, overwritten each fill.
lsa
Log Sequence Address of the staged page.
”Newest wins” when two copies of a VPID exist.
ensure_metadata
If true, syncing volume also flushes metadata.
Volume headers need destination metadata durable.
position_in_block
Slot index (0 .. num_block_pages-1). Fixed.
Fixes offset in write_buffer (position_in_block * IO_PAGESIZE).
block_no
Owning block’s index in dwb_Global.blocks. Fixed.
Lets a bare DWB_SLOT* find its block.
INVARIANT (slot↔buffer aliasing). For every block and slot index i, block->slots[i].io_page == (FILEIO_PAGE *)(block->write_buffer + i * IO_PAGESIZE). Established in dwb_create_blocks; if violated, the single block write of write_buffer no longer carries that page, silently dropping it from the DWB volume:
FLUSH_VOLUME_INFO array, one per distinct dest volume.
Pages span several home volumes; fsync each once.
count_flush_volumes_info
Entries in use. volatile.
Grows as new dest volumes appear; read by file-sync helper.
max_to_flush_vdes
Capacity (== num_block_pages).
Worst case: each page a different volume.
mutex
Protects this block’s wait_queue.
Serializes threads parking on flush.
wait_queue
Threads waiting for this block to flush.
Backpressure (Ch 8).
write_buffer
The only owned page memory — num_block_pages * IO_PAGESIZE.
Block written in one I/O; slots[i].io_page alias into it.
slots
num_block_pagesDWB_SLOT, a view over write_buffer.
Per-page metadata + per-home write handle.
count_wb_pages
Fill level. volatile.
At num_block_pages the block is full; bumped atomically.
block_no
Index in dwb_Global.blocks.
Self-id; matches every slot’s block_no.
version
Counter bumped per flush. volatile.
Distinguishes incarnations; via DWB_GET_BLOCK_VERSION.
all_pages_written
True once every page reached home. volatile.
Tells the driver the block is drained/reusable.
INVARIANT (fill monotonicity within a version). Within one version, count_wb_pages only rises from 0 to num_block_pages (reset with a version bump on recycle); producers advance the cursor atomically (Ch 4), so backward motion without a bump would hand two producers the same slot.
flowchart LR
BLK["double_write_block"] --> WB["write_buffer<br/>num_block_pages * IO_PAGESIZE"]
BLK --> SLOTS["slots[] — VIEW onto write_buffer"]
BLK --> FVI["flush_volumes_info[] — one per home volume"]
Figure 1-2. A block’s three parallel arrays; only slots aliases write_buffer.
Not yet fsync’d for this block. Initial state at allocation.
VOLUME_FLUSHED_BY_DWB_FILE_SYNC_HELPER_THREAD
The file-sync helper won the CAS and fsync’d.
VOLUME_FLUSHED_BY_DWB_FLUSH_THREAD
The flush thread fsync’d (helper didn’t get there first).
INVARIANT (fsync exactly once per volume per block). Flush thread and file-sync helper race on ATOMIC_CAS_32 (&...flushed_status, VOLUME_NOT_FLUSHED, ...); the winner fsyncs, the loser skips, and the CAS doubles as the durability/handoff signal (Ch 6–7).
1.5 dwb_slots_hash_entry and the lock-free hash map
Brief per-entry critical sections; owned by ctor/dtor.
del_id
Deletion txn id for safe reclamation.
Defers freeing until no reader can hold it.
slot
The DWB_SLOT holding this VPID’s page.
Resolves VPID to staged page for reads.
slots_entry_Descriptor registers these by offset — stack (freelist), next (bucket), del_id (reclaim), vpid (key), mutex, LF_EM_USING_MUTEX — how the generic map manipulates the opaque nodes. The typedef binds it:
using dwb_hashmap_type = cubthread::lockfree_hashmap<VPID, dwb_slots_hash_entry>;
INVARIANT (hash entry tracks the newest staged copy). At most one live entry per VPID, its slot pointing at the newest staged image (highest lsa); re-adding repoints the entry so dwb_read_page never returns a stale image.
double_write_wait_queue_entry: data — the parked thread’s wakeup token (so the waker signals the right waiter); next — link chaining waiters FIFO from head to tail.
double_write_wait_queue:
Field
Role
Why it exists
head
First waiter (dequeue end).
FIFO: oldest woken first.
tail
Last waiter (enqueue end).
O(1) append.
free_list
Recycled entry nodes.
Avoids malloc churn.
count
Live waiters.
Fast size check
free_count
Nodes on free_list.
Observes the recycle pool.
Queues operate under the owning struct’s mutex; teardown is Ch 10.
1.7 double_write_buffer (the dwb_Global singleton)
Hand-off to helper; CAS’d to NULL when consumed (ATOMIC_TAS_ADDR).
INVARIANT (singleton consistency). The constructor zero/NULL-inits everything (vdes (NULL_VOLDES), position_with_flags (0)), so a no-DWB process still has a valid dwb_Global; dwb_is_created reads position_with_flags, not blocks. For a created buffer, num_pages == num_blocks * num_block_pages and num_block_pages == 1 << log2_num_block_pages, all powers of two — Chapter 2’s shift/mask decoding is correct only under these relations, set once at creation.
One real buffer, two views. A block owns one write_buffer; slots[] and flush_volumes_info[] are parallel arrays over it, not page copies.
The aliasing invariant is load-bearing.slots[i].io_page == write_buffer + i*IO_PAGESIZE, wired in dwb_create_blocks/dwb_initialize_slot keyed by position_in_block; the design collapses if it breaks.
DWB_SLOT is a borrowed handle.io_page is never freed through the slot; vpid/lsa/ensure_metadata rewritten on reuse, position_in_block/block_no fixed at init.
Per-volume durability is a tri-state race. The CAS on flushed_status lets exactly one of {flush thread, file-sync helper} fsync each home volume.
The hash map is a freshness index, not a copy.slot points at the newest staged image; stack/next/del_id/mutex exist only for the offset descriptor and reclamation.
position_with_flags is the lock-free protocol in one word. Producers and the flush driver coordinate through this single atomic; Chapter 2 decodes it.
dwb_Global is always present. The constructor leaves it consistent; existence is read from position_with_flags, and the power-of-two relations hold for any created buffer.
The entire concurrency contract of the double write buffer rides on one 64-bit
field: dwb_Global.position_with_flags. Slot allocation, per-block “write
started” tracking, and the create / structure-modify lifecycle all share that
word, mutated only through ATOMIC_CAS_64 and read only through
ATOMIC_INC_64 (&..., 0ULL) (an atomic add-of-zero used as a torn-read-free
load). This chapter walks the bit layout, every accessor and mutator macro, and
the two CAS retry loops that advance it. For why a single atomic replaces
separate counters and locks, see the high-level companion’s “Position-with-flags
— the central coordinator” (cubrid-double-write-buffer.md).
Monotonic ring cursor into the flattened slot array; advanced by DWB_GET_NEXT_POSITION_WITH_FLAGS, wraps at num_pages.
30
DWB_CREATE (0x40000000)
DWB exists / usable
Set by DWB_STARTS_CREATION (creation), cleared by DWB_ENDS_CREATION (teardown).
31
DWB_MODIFY_STRUCTURE (0x80000000)
resize in progress
Set by DWB_STARTS_MODIFYING_STRUCTURE (resize start), cleared by DWB_ENDS_MODIFYING_STRUCTURE; blocks slot allocation while set.
32..63
block-status bitmask
DWB_BLOCKS_STATUS_MASK (0xffffffff00000000)
One bit per block (block b at bit 63 - b); set means “writing started”. Set by DWB_STARTS_BLOCK_WRITING, cleared by DWB_ENDS_BLOCK_WRITING.
DWB_FLAG_MASK is everything except the 30-bit position — the bits that must
survive a position increment (2.5). Block b’s status bit at (63 - b) puts
block 0 at bit 63 down to block 31 at bit 32, capping the design at 32 blocks
(DWB_MAX_BLOCKS, against DWB_MIN_BLOCKS1) because there are only 32 status
bits. (The title’s “BLOCKS_FLUSH” is informal: there is noBLOCKS_FLUSH flag
in this word — the 32-bit region is the “write started” bitmask, and the separate
blocks_flush_counter integer lives outside position_with_flags.)
Invariant — the three regions never overlap. Bit-disjoint by construction of
the masks. The position maxes at 0x3fffffff (over one billion), far above any
legal num_pages (DWB caps at 32 MB, Chapter 3), so position arithmetic never
carries into bit 30. Were the masks to overlap, a position increment would
silently clear a lifecycle or status bit and let a producer overwrite a block
still being flushed. Guaranteed by the constants alone, not re-checked at runtime.
flowchart LR
subgraph W["position_with_flags (UINT64)"]
direction LR
B["bits 63..32\nblock-status bitmask\nblock b -> bit 63-b"]
M["bit 31\nDWB_MODIFY_STRUCTURE"]
C["bit 30\nDWB_CREATE"]
P["bits 29..0\nslot position\nDWB_POSITION_MASK"]
end
Figure 2-1. Bit partition of position_with_flags.
2.2 Position accessors: GET, RESET, and the block-status reader
// position projections -- src/storage/double_write_buffer.cpp
#defineDWB_GET_POSITION(position_with_flags)\
((position_with_flags) & DWB_POSITION_MASK) /* <- bits 0..29 only */
#defineDWB_RESET_POSITION(position_with_flags)\
((position_with_flags) & DWB_FLAG_MASK) /* <- keep flags, zero position */
#defineDWB_GET_BLOCK_STATUS(position_with_flags)\
((position_with_flags) & DWB_BLOCKS_STATUS_MASK) /* <- bits 32..63 only */
DWB_GET_POSITION returns the bare ring cursor; DWB_RESET_POSITION is its
complement, used at creation (2.6) to start a freshly built structure from
position 0 while keeping the incoming flags. DWB_GET_BLOCK_STATUS returns the
raw block-status word (high 32 bits); callers never decode it bit-by-bit — both
of its uses are whole-region zero-checks: a post-flush
assert (DWB_GET_BLOCK_STATUS (...) == 0) at the end of
dwb_starts_structure_modification (every block must be drained before a resize
proceeds) and an “is anything to flush?” early-out if (DWB_GET_BLOCK_STATUS (...) == 0)
in dwb_flush_force. Per-bit testing goes through DWB_IS_BLOCK_WRITE_STARTED
(2.4) instead.
2.3 Decoding a position into block-number and in-block offset
A raw position is a flat index across all slots. Since the per-block page count
is a power of two, the block number is the high part and the in-block offset the
low part, split at log2_num_block_pages. The two decode macros:
// position decode -- src/storage/double_write_buffer.cpp
Shifting right by that log gives the block index; masking with
DWB_BLOCK_NUM_PAGES - 1 (an all-ones low mask, valid because num_block_pages
is a power of two — Chapter 3) gives the in-block offset. The unsigned int cast
is lossless (the masked position fits in 30 bits) and keeps the shift in 32-bit
arithmetic. The allocation loop reads both, then asserts
current_block_no < DWB_NUM_TOTAL_BLOCKS && position_in_current_block < DWB_BLOCK_NUM_PAGES.
DWB_IS_BLOCK_WRITE_STARTED(w,b) tests bit 63-b,
DWB_IS_ANY_BLOCK_WRITE_STARTED(w) the whole region;
DWB_STARTS_BLOCK_WRITING / DWB_ENDS_BLOCK_WRITING return a word with that bit
set / cleared. Three wrap an assert via the comma operator (assert first, then
the bit computation as the value): the STARTS/IS pair guards
block_no < DWB_MAX_BLOCKS (out of range would shift into the position/flag
region), ENDS asserts the bit is already set, catching a double-clear. All four
are pure, leaving the CAS to the caller.
The producer that grabs a block’s first slot must set the STARTS bit and
advance the cursor in the same published word. The allocator builds that
composite before the CAS — DWB_STARTS_BLOCK_WRITING first, then
DWB_GET_NEXT_POSITION_WITH_FLAGS (2.5) on top of it, so one CAS publishes both:
DWB_STARTS_BLOCK_WRITING (current_position_with_flags, current_block_no); /* <- set status bit */
new_position_with_flags =
DWB_GET_NEXT_POSITION_WITH_FLAGS (current_position_with_block_write_started); /* <- then advance cursor */
Invariant — a block’s status bit stays set for one flush cycle. Set when the
block’s first slot is allocated (DWB_STARTS_BLOCK_WRITING), cleared only after
the block is flushed (DWB_ENDS_BLOCK_WRITING, Chapter 6). While set, an
allocator wrapping back to that first slot (position_in_current_block == 0)
sees DWB_IS_BLOCK_WRITE_STARTED true and waits:
// ... condensed: ER_CSS_PTHREAD_COND_TIMEDOUT also retries ...
goto start; /* <- retry the whole allocation after the wait */
}
If this bit were cleared early (or lost to an overlapping mask), a producer would
overwrite a block whose pages have not reached the on-disk DWB volume, defeating
the torn-page protection.
2.5 The next-position macro and the allocation CAS loop
Two branches (Figure 2-2): the wrap branch at num_pages - 1 returns
position_with_flags & DWB_FLAG_MASK (position to 0, flags kept); otherwise the
increment branch returns position_with_flags + 1, which touches only the
low bits since the position is well below bit 30.
flowchart TD
S["DWB_GET_NEXT_POSITION_WITH_FLAGS(w)"] --> Q{"DWB_GET_POSITION(w) == num_pages - 1 ?"}
Q -->|yes, last slot| WRAP["return w AND DWB_FLAG_MASK\nposition -> 0, flags kept"]
Q -->|no| INC["return w + 1\nposition advances, flags untouched"]
Figure 2-2. The two branches of DWB_GET_NEXT_POSITION_WITH_FLAGS.
Invariant — flag bits survive both increment and wrap.DWB_FLAG_MASK is
the exact complement of DWB_POSITION_MASK, so the create, structure-modify, and
status bits emerge unchanged either way — which lets one CAS atomically advance
the cursor and publish the status bit set just above, both in the same word.
Had the wrap branch returned a bare 0, it would erase DWB_CREATE and every
status bit, so producers would think the buffer gone.
This macro feeds the allocation CAS loop in dwb_acquire_next_slot, the
first of the two retry loops on this word. The word is read with
ATOMIC_INC_64 (&dwb_Global.position_with_flags, 0ULL), the early-out and
first-slot/non-first-slot branches (2.4) build new_position_with_flags, and a
single CAS commits it; a lost CAS goto starts to reload:
The creation trio acts on bit 30, the modify trio on bit 31: the set macros are
a plain OR, the clear macros assert the bit is set (catching unbalanced
begin/end pairs) then AND with the complement.
DWB_NOT_CREATED_OR_MODIFYING masks both lifecycle bits and tests whether the
result is exactlyDWB_CREATE — false only for “created and not modifying”,
true if the DWB does not exist (bit 30 clear) or a resize is underway (bit 31
set). It is the early-out guard atop the allocation and flush paths
(if (DWB_NOT_CREATED_OR_MODIFYING (...)) { ... skip ... }). When it fires in
dwb_acquire_next_slot the guard fans out three ways: DWB_IS_MODIFYING_STRUCTURE
true waits via dwb_wait_for_strucure_modification then goto starts; otherwise
the DWB was deleted, returning ER_DWB_DISABLED if any block still shows a started
write (DWB_IS_ANY_BLOCK_WRITE_STARTED — data was lost) and a quiet NO_ERROR
if not; the final else is assert (false) (the masked compare excludes it).
Creation publishes DWB_CREATE via DWB_RESET_POSITION → DWB_STARTS_CREATION
→ CAS, starting the new structure at position 0 with the create bit lit
(assert(false) on failure — creation is single-threaded):
if (!ATOMIC_CAS_64 (&dwb_Global.position_with_flags, *current_position_with_flags, new_position_with_flags))
{
assert (false); /* <- single-threaded; a race here is a bug */
}
The modify-structure CAS loop in dwb_starts_structure_modification is the
second (and last) genuine retry loop on this word. It reads the word atomically,
bails if a resize is already in flight (only one resizer allowed — note this
returns ER_FAILED instead of waiting), ORs in DWB_MODIFY_STRUCTURE, and spins
on CAS until it wins:
while (!ATOMIC_CAS_64 (&dwb_Global.position_with_flags, local_current_position_with_flags, new_position_with_flags));
// ... condensed: drain in-flight flushers, flush all started blocks, then assert status==0 ...
Invariant — one masked compare distinguishes “open for business” from every
other state.DWB_NOT_CREATED_OR_MODIFYING checks bit 31 together with bit
30: a thread that set DWB_MODIFY_STRUCTURE without clearing DWB_CREATE (exactly
what the loop above does) is still excluded, since the masked value is then
DWB_CREATE | DWB_MODIFY_STRUCTURE, which is != DWB_CREATE. In a separate word, a
producer could see a stale “created” state and allocate into a half-rebuilt
buffer. The same single-resizer guarantee is what makes the unconditional
DWB_STARTS_MODIFYING_STRUCTURE OR safe: the DWB_IS_MODIFYING_STRUCTURE check
inside the loop body, re-validated by the CAS, ensures no two threads both win
the set.
dwb_Global.position_with_flags is one UINT64 split into three bit-disjoint
regions: a 30-bit ring position (DWB_POSITION_MASK), two lifecycle flags
(DWB_CREATE bit 30, DWB_MODIFY_STRUCTURE bit 31), and a 32-bit per-block
“write started” bitmask (DWB_BLOCKS_STATUS_MASK) in the top word. It is read
with ATOMIC_INC_64 (&..., 0ULL) and written only by ATOMIC_CAS_64.
Block b’s status bit lives at (63 - b), hard-capping blocks at
DWB_MAX_BLOCKS = 32; a raw position decodes to (block_no, position_in_block)
by splitting at log2_num_block_pages (shift right / mask num_block_pages - 1),
relying on power-of-two sizing.
DWB_GET_NEXT_POSITION_WITH_FLAGS wraps at num_pages - 1 to
w & DWB_FLAG_MASK (position to 0, flags kept), else w + 1; either way the
flag/status bits survive, letting one CAS publish a cursor advance and a
status-bit set atomically — the allocator builds DWB_STARTS_BLOCK_WRITING
then DWB_GET_NEXT_POSITION_WITH_FLAGS on top of it before committing.
Two CAS retry loops mutate this word: the allocation loop in
dwb_acquire_next_slot (goto start on lost CAS, plus first-slot wait via
dwb_wait_for_block_completion and modify/not-created early-outs), and the
do/while (!ATOMIC_CAS_64) loop in dwb_starts_structure_modification that
sets DWB_MODIFY_STRUCTURE and returns ER_FAILED if a resize is already in
flight. Creation uses a single non-retrying CAS (assert(false) on loss).
The block-status bit is set when a block’s first slot is claimed, cleared only
after the block is flushed; the clear/end macros assert the matching “is set”
predicate first, and all mutator macros are pure, leaving the commit to a
caller CAS.
DWB_NOT_CREATED_OR_MODIFYING collapses both lifecycle bits into one masked
compare against DWB_CREATE, gating the fast paths out whenever the DWB is
absent or resizing; DWB_GET_BLOCK_STATUS is used only for whole-region
zero-checks (post-drain assert in the resize path, “anything to flush?” in
dwb_flush_force), never decoded bit-by-bit.
Chapter 3: Parameter Loading Block Allocation and Memory Layout
Reader question: given the structs of Chapter 1 and the position word of Chapter 2, how is the DWB sized, allocated, formatted on disk, and wired into boot? We trace every branch from the two tunables (PRM_ID_DWB_SIZE, PRM_ID_DWB_BLOCKS) to the malloc’d write buffers, the slot/io_page aliasing, on-disk formatting, and teardown. Recovery-time block reconstruction is out of scope (Chapter 8). For why double-writing exists see cubrid-double-write-buffer.md § “Why a double write buffer”.
Both tunables are clamped to a power of two so num_pages = size / IO_PAGESIZE and num_block_pages = num_pages / num_blocks are exact powers of two — letting the runtime cache log2_num_block_pages (§3.7) for a mask shift instead of a modulo. Three functions cooperate: dwb_power2_ceil, dwb_load_buffer_size, dwb_load_block_count.
if (*p_value < min) { *p_value = min; } /* branch A: below floor -> snap to min */
elseif (*p_value > max) { *p_value = max; } /* branch B: above ceiling -> snap to max */
elseif (!IS_POWER_OF_2 (*p_value)) { /* branch C: in range, not power of 2 */
limit = min <<1;
while (limit <*p_value) limit = limit <<1; /* shift up until >= value -> round UP */
*p_value = limit;
}
/* branch D (implicit): in range AND already power of 2 -> untouched */
assert (IS_POWER_OF_2 (*p_value));
assert (*p_value >= min &&*p_value <= max);
A/B clamp to a boundary (both constants are powers of two); C rounds up, starting at min << 1 and only shifting left so it cannot overshoot max; D leaves an already-power-of-two value untouched.
Invariant 3-A (power-of-two within clamp). After return, *p_value is a power of two in [min, max], enforced by the asserts. If violated, num_block_pages would not be a power of two and log2_num_block_pages would no longer be a valid shift width.
flowchart TD
S["dwb_power2_ceil(min,max,*v)"] --> A{"*v < min?"}
A -->|yes| AA["*v = min"] --> Z["assert power-of-2 in range"]
A -->|no| B{"*v > max?"}
B -->|yes| BB["*v = max"] --> Z
B -->|no| C{"IS_POWER_OF_2?"}
C -->|yes| Z
C -->|no| CC["limit=min<<1; while limit<*v: limit<<=1; *v=limit"] --> Z
Figure 3-2-1 — the four branches of dwb_power2_ceil
3.3 dwb_load_buffer_size and dwb_load_block_count — the 0-disables gate
Read the parameter, treat 0 as “disable DWB” (return false), else clamp via dwb_power2_ceil and return true. dwb_load_block_count is structurally identical against PRM_ID_DWB_BLOCKS / DWB_MIN_BLOCKS / DWB_MAX_BLOCKS.
Invariant 3-B (0 means off, atomically). A 0 in either parameter disables the buffer. The short-circuit || in the caller (!dwb_load_buffer_size(...) || !dwb_load_block_count(...)) means if size is 0 the block-count parameter is never read, and the function returns NO_ERROR with nothing allocated — no half-initialized state.
if (io_page !=NULL) { /* defensive; always true here */
VPID_SET (&slot->vpid, io_page->prv.volid, io_page->prv.pageid); /* seed from page header */
LSA_COPY (&slot->lsa, &io_page->prv.lsa);
}
slot->position_in_block = position_in_block;
slot->block_no = block_no;
The full double_write_slot has six fields; this sets five (the sixth, ensure_metadata, stays at its memset-0):
Field
Role at init / Why it exists
io_page
Caller-supplied alias into the block write buffer (§3.6) — the slot’s window onto the page bytes, enabling a single-page write
vpid
Seeded from io_page->prv.{volid,pageid} (both -1 at create) — identifies the home page; rewritten on each stage
lsa
Copied from io_page->prv.lsa (NULL_LSA at create) — ordering and the slots-hash freshness check
ensure_metadata
Not set here; stays 0 from the dwb_create_blocksmemset. Flags that block metadata (volume header) must be fsynced on flush; set later at stage/flush time (Ch 4–6)
position_in_block
j, index within the block — stable identity; maps the slot to its IO_PAGESIZE offset
block_no
i, owning block index — back-pointer to the block
At create vpid/lsa are placeholders (header just zeroed by fileio_initialize_res), overwritten when a real page is staged (Chapter 4).
Per-block FLUSH_VOLUME_INFO[num_block_pages] array — one entry per distinct home volume touched in a flush
count_flush_volumes_info
0 — live count of volumes accumulated this fill cycle
max_to_flush_vdes
num_block_pages — hard ceiling; sized so worst case every page hits a distinct volume
mutex
Freshly inited — protects this block’s wait queue and counters
wait_queue
Empty — threads waiting for this block’s flush park here
write_buffer
The contiguous num_block_pages * IO_PAGESIZE malloc — backs a single bulk write
slots
The DWB_SLOT[num_block_pages] array — per-page descriptors aliasing into write_buffer
count_wb_pages
0 — pages staged so far; the fill cursor
block_no
i — the block identity used everywhere
version
0 — monotonic flush-generation counter (Ch 2)
all_pages_written
false — set true once fully staged and ready to flush
Invariant 3-C (flush-info array over-provisioned).max_to_flush_vdes == num_block_pages guarantees count_flush_volumes_info never exceeds bounds even if every page targets a different volume. The runtime appends to flush_volumes_info[count_flush_volumes_info++] with no bounds check, relying on this sizing.
3.6 dwb_create_blocks — the allocation engine and the aliasing trick
Allocate the block array, then per block a contiguous write buffer, a slots array, and a flush-volumes array; then alias each slot’s io_page into the write buffer.
dwb_initialize_block (&blocks[i], i, 0, blocks_write_buffer[i], slots[i], flush_volumes_info[i], 0, num_block_pages);
}
*p_blocks = blocks;
return NO_ERROR;
exit_on_error:
for (i =0; i < DWB_MAX_BLOCKS; i++) { /* free slots[i], blocks_write_buffer[i], flush_volumes_info[i] if non-NULL */ }
if (blocks !=NULL) free_and_init (blocks);
return error_code;
}
The load-bearing move is pass 4’s aliasing: each slots[i][j].io_page is write_buffer + j * IO_PAGESIZE, a pointer into the contiguous buffer, so the block flushes in one bulk write yet stays individually addressable. fileio_initialize_res zeroes each FILEIO_PAGE header (pageid/volid = -1, NULL LSA, zero ptype/pflag/reserves/tde_nonce) as clean-slate hygiene; it is overwritten before flushing. Figure 3-6-1 traces the branches; the scratch arrays sized DWB_MAX_BLOCKS are NULL-inited so the error funnel frees them uniformly and *p_blocks is assigned only on full success.
Invariant 3-D (one block = one contiguous buffer). Every slots[i][j].io_page points strictly inside blocks_write_buffer[i] at offset j*IO_PAGESIZE, making block->write_buffer a valid single I/O vector. Malloc’ing pages separately would break the bulk DWB-volume write in Chapter 6.
flowchart TD
A["malloc blocks[]"] -->|NULL| ERR["exit_on_error"]
A -->|ok| B["pass1: malloc write_buffer per block"]
B -->|any NULL| ERR
B -->|ok| C["pass2: malloc slots per block"]
C -->|any NULL| ERR
C -->|ok| D["pass3: malloc flush_volumes_info per block"]
D -->|any NULL| ERR
D -->|ok| E["pass4: alias io_page, init slot, init block"]
E --> F["*p_blocks = blocks; return NO_ERROR"]
ERR --> G["free all non-NULL scratch[i], free blocks, return error_code"]
Figure 3-6-1 — dwb_create_blocks allocation passes and the single error funnel
3.6.1 dwb_finalize_block — the per-block teardown counterpart
Because the slots’ io_page alias into write_buffer, there is no separate free of the pages — freeing write_buffer reclaims all num_block_pages at once (the destruction-side mirror of Inv 3-D). Wait queue drained and mutex destroyed last.
3.7 dwb_create_internal — sizing, formatting, wiring the global
Three load-bearing points: the disable gate returns NO_ERROR touching nothing if either loader returns false; volume-before-blocks, the LOG_DBDWB_VOLID volume is formatted and prior dirties synced before any block exists; the publishing CAS is the barrier (caller holds the modification flag, so assert(false) would fire only on a logic bug). The fourth assert bounds the §3.6 scratch; slots_entry_Descriptor is the lock-free hash’s vtable, detailed where the slots hash is exercised.
Invariant 3-E (geometry published atomically with the CREATE flag). All dwb_Global geometry is written before the single ATOMIC_CAS_64 that sets the CREATE flag. Readers gate on DWB_IS_CREATED(position_with_flags); until the CAS lands they see “not created” and never observe a half-filled dwb_Global. The CAS is the publication barrier.
Error path.exit_on_error dismounts and unformats the volume if vdes was opened, then if blocks was allocated walks 0 .. num_blocks-1 calling dwb_finalize_block and frees the array. The position word is untouched, so the global still reads “not created”.
3.8 dwb_create and fileio_make_dwb_name — the public wrapper and the name
dwb_ends_structure_modification (thread_p, current_position_with_flags); /* always release the flag */
return error_code;
Every path — already-created, internal failure, success — funnels through end: so the modification flag is always released (Chapter 10). The name is a plain sprintf:
FILEIO_SUFFIX_DWB is "_dwb", so db demodb under /db/log yields /db/log/demodb_dwb. The id is the fixed LOG_DBDWB_VOLID = LOG_DBFIRST_VOLID - 22 — a reserved system slot, not a data volume.
Boot wire-up.boot_create_database calls dwb_createbefore the first data volume:
The inverse of §3.7, in reverse order: wait queue and global mutex, then every block via dwb_finalize_block (iterating DWB_NUM_TOTAL_BLOCKS), the hashmap, the volume (dismount + unformat deletes the file), and finally the CAS that clears the CREATE flag under the modification flag (Inv 3-E in reverse). After return the global reads “not created” and the _dwb file is gone.
dwb_power2_ceil clamps-and-rounds each tunable to a power of two through four branches (below-floor, above-ceiling, round-up, untouched) so the geometry divides exactly — Inv 3-A.
A 0 in either loader disables the DWB atomically via the short-circuit || — Inv 3-B.
dwb_create_blocks aliases each slots[i][j].io_page into one contiguous write_buffer, giving a bulk write plus per-page addressing; dwb_finalize_block frees the buffer once — Inv 3-D.
Each flush_volumes_info is sized num_block_pages so the unchecked append never overflows; ensure_metadata stays 0 from the memset until stage/flush time — Inv 3-C.
dwb_create_internal formats the LOG_DBDWB_VOLID volume and syncs prior dirties before building blocks, then publishes geometry with one ATOMIC_CAS_64 — Inv 3-E.
fileio_make_dwb_name builds <log_path>/<db>_dwb; boot_create_database calls dwb_create before disk_format_first_volume.
dwb_destroy_internal reverses §3.7 and clears the CREATE flag with a CAS, restoring “not created”.
This chapter answers one producer-side question: how does a thread claim the next free DWB slot without a lock, copy its page bytes in, and what does it do on every contention and lifecycle branch? The lock-free claim is the heart of DWB’s write path — see the companion’s “Lock-free position cursor” for the why; here we trace the how, branch by branch.
The call chain is dwb_set_data_on_next_slot (wrapper) → dwb_acquire_next_slot (lock-free CAS claim, may block in dwb_wait_for_strucure_modification or dwb_wait_for_block_completion) → dwb_set_slot_data (memcpy the page in). The position-with-flags word and its macros are from Chapter 2.
FILEIO_PAGE *io_page; /* The contained page or NULL. */
VPID vpid; /* The page identifier. */
LOG_LSA lsa; /* The page LSA */
bool ensure_metadata; /* Include metadata when syncing */
unsignedint position_in_block;/* The position in block. */
unsignedint block_no; /* The number of the block where the slot reside. */
};
Field
Role
Why it exists / who sets it
io_page
Pointer into the block’s staging buffer — destination of the page copy.
Pre-wired at block allocation (Ch 3) to a fixed offset, never reassigned. dwb_set_slot_datamemcpys the producer’s page into the bytes it points at.
vpid
The staged page’s identity and the slot’s visibility flag.
Nulled at claim, re-set last by dwb_set_slot_data (Invariant 4-C). A non-NULL vpid is the only signal a reader uses to decide the slot holds a complete image.
lsa
The page’s log-sequence address, captured at staging.
Lets flush/recovery (Ch 9) order or look the slot up by LSA without re-parsing the page header. Copied from io_page_p->prv.lsa.
ensure_metadata
Whether the eventual sync must also flush volume metadata.
Carried verbatim from the producer’s arg (Ch 6–7 act on it). Set in dwb_set_slot_data.
position_in_block
The slot’s fixed index in its block, 0 .. DWB_BLOCK_NUM_PAGES-1.
Pre-wired at allocation, immutable; the claim asserts it equals the cursor-derived offset (Invariant 4-D) — the slot↔DWB-volume-offset binding, not recomputed per write.
block_no
Which block the slot belongs to.
Pre-wired at allocation; lets later stages recover the owning block from a bare slot pointer.
The acquisition path writes only vpid (nulled); staging writes io_page’s contents plus lsa, vpid, ensure_metadata. position_in_block and block_no are read-only.
error_code != NO_ERROR propagates verbatim; *p_dwb_slot == NULL after success is the can’t-but-not-broken outcome, allowed only for a non-waiter.
Invariant 4-A — a waiter always leaves with a slot. When can_wait == true, this function returns a non-NULL *p_dwb_slot (or a hard error); dwb_acquire_next_slot enforces it by goto start-ing after every wait, so the only NULL-with-NO_ERROR exits are on can_wait == false arms. The assert traps a waiter returning NULL — in release that would silently drop a guaranteed page write.
A read-modify-CAS retry loop on start:, wrapping a lifecycle/contention preamble. It opens by nulling the out-param (*p_dwb_slot = NULL), so every early return NO_ERROR arm leaves the caller with NULL. Figure 4-1 is the branch map; the prose walks each arm.
flowchart TD
A["start: atomic read pos"] --> B{NOT_CREATED_OR_MODIFYING?}
B -- "no, normal" --> P{position_in_block == 0?}
B -- "yes, rare" --> C{MODIFYING_STRUCTURE?}
C -- "yes" --> C1{can_wait?}
C1 -- "false" --> CR["return NO_ERROR, slot NULL"]
C1 -- "true" --> C2["wait_for_strucure_modification"]
C2 --> C3{rc?}
C3 -- "TIMEDOUT or NO_ERROR" --> A
C3 -- "other error" --> CE["return error_code"]
C -- "no" --> D{!IS_CREATED?}
D -- "yes" --> D1{ANY_BLOCK_WRITE_STARTED?}
D1 -- "yes" --> DE["ER_DWB_DISABLED"]
D1 -- "no" --> DR["return NO_ERROR, slot NULL"]
D -- "no, impossible" --> DX["assert false"]
P -- "yes, first writer" --> E{BLOCK_WRITE_STARTED?}
E -- "yes" --> E1{can_wait?}
E1 -- "false" --> ER["return NO_ERROR, slot NULL"]
E1 -- "true" --> E2["wait_for_block_completion"]
E2 --> E3{rc?}
E3 -- "TIMEDOUT or NO_ERROR" --> A
E3 -- "other error" --> EE["return error_code"]
E -- "no" --> F["new = NEXT(STARTS_BLOCK_WRITING)"]
P -- "no, mid-block" --> G["new = NEXT(pos)"]
F --> H{ATOMIC_CAS_64 ok?}
G --> H
H -- "no, lost race" --> A
H -- "yes" --> I["claim slot, null vpid, return"]
Figure 4-1. Every branch of dwb_acquire_next_slot.
Every pass re-reads the cursor at the start: label via ATOMIC_INC_64 (&dwb_Global.position_with_flags, 0ULL) — add-zero is the idiom for an atomic load. Re-reading on every retry makes the loop correct: any goto start re-observes the cursor, so a thread that lost a CAS, woke from a wait, or saw a transient flag never acts on stale bits.
4.3.1 The rare lifecycle preamble: DWB_NOT_CREATED_OR_MODIFYING
(a) Modifying structure.ER_CSS_PTHREAD_COND_TIMEDOUT is not propagated — it becomes goto start; any other error (alloc failure, ER_INTERRUPTED on shutdown) returns; a clean wake also goto start-s, since it only means “modification ended” and the cursor must be re-read.
(b) Not created. If any block still has its write-started bit set, that is an inconsistent partial teardown → ER_DWB_DISABLED; otherwise the DWB was simply turned off and the caller writes the page directly.
(c) Dead arm.DWB_NOT_CREATED_OR_MODIFYING is (pos & DWB_CREATE_OR_MODIFY_MASK) != DWB_CREATE; the two if arms exhaust the ways that holds, so the else is unreachable. There is no return after the assert — in release, control falls through into the normal path with an abnormal cursor, so it is strictly a debug guard.
4.3.2 Decoding the cursor and the first-into-block branch
Past the preamble, the cursor is decoded into a block number (DWB_GET_BLOCK_NO_FROM_POSITION) and a within-block offset (DWB_GET_POSITION_IN_BLOCK), each bounds-asserted against DWB_NUM_TOTAL_BLOCKS / DWB_BLOCK_NUM_PAGES. The decision turns on whether this is the first producer into the block.
The first producer sets the started bit only after confirming it is clear. If still set, the previous fill is unflushed and reusing its slots would overwrite data the flush still needs (the wrap-around hazard, rare except in a single-block buffer): can_wait == false bails, a waiter calls dwb_wait_for_block_completion (§4.4) and goto start-s on a clean wake. When clear, DWB_STARTS_BLOCK_WRITING ORs in the status bit and DWB_GET_NEXT_POSITION_WITH_FLAGS advances the position — both in one 64-bit word, so the CAS publishes them atomically.
position_in_current_block != 0 — a mid-block writer. The else arm only advances the position (new = DWB_GET_NEXT_POSITION_WITH_FLAGS(pos)) under two debug asserts, DWB_IS_CREATED(pos) and !DWB_IS_MODIFYING_STRUCTURE(pos).
Invariant 4-B — a non-first writer cannot race teardown. Once any thread has claimed slot 0, the block’s started bit pins the DWB created and non-modifying until that block is flushed; a thread seeing position_in_current_block != 0 may assume both flags hold (the asserts enforce it in debug). If violated, a producer could stage into a block being torn down.
4.3.3 The CAS commit and post-claim initialization
The CAS is the linearization point: it succeeds only if no one mutated the cursor since the read. On failure the thread goto start-s from a fresh read (so a lost first-writer race does not re-set the started bit). On success it owns the slot exclusively (no two CAS winners for one cursor value) and nulls vpid.
Invariant 4-D — slot index maps 1:1 to a DWB-volume offset. The claimed slot is blocks[block_no].slots + position_in_block, with slots[i].position_in_block == i pre-wired at block build (Ch 3); the closing assert re-checks it against the cursor-derived offset. That index is also the slot’s fixed offset into the DWB volume, so a slot acquired at cursor position p always writes the same on-disk location — the producer never chooses a destination, the cursor does.
Both wait helpers are server-mode only (in !SERVER_MODE, no-ops returning NO_ERROR) and share one shape: lock a mutex, recheck under it, enqueue, suspend with a timeout, classify the resume.
dwb_wait_for_strucure_modification (the original misspells “structure” — it is the real symbol name) waits on the globaldwb_Global.wait_queue under dwb_Global.mutex with a 10 ms timeout:
r =thread_suspend_timeout_wakeup_and_unlock_entry (thread_p, &to, THREAD_DWB_QUEUE_SUSPENDED); /* <- 10 ms */
The recheck under the mutex guards against sleeping on an already-cleared condition. Three resume arms: ER_CSS_PTHREAD_COND_TIMEDOUT → remove entry, return timeout (caller retries); resume_status != THREAD_DWB_QUEUE_RESUMED → shutdown wake, ER_INTERRUPTED; else → woken by dwb_signal_structure_modificated, NO_ERROR.
dwb_wait_for_block_completion is the same shape but on the per-blockdwb_block->wait_queue under dwb_block->mutex, rechecks the block’s started bit via DWB_IS_BLOCK_WRITE_STARTED, and uses a 20 ms timeout. The two variants differ in when they take the per-thread lock relative to the recheck: block-completion locks dwb_block->mutex, then thread_lock_entry, then rechecks (so it holds both locks across the recheck); structure-mod locks dwb_Global.mutex, rechecks DWB_IS_MODIFYING_STRUCTUREfirst, and only calls thread_lock_entry once it has decided to enqueue. Either way the recheck-under-mutex prevents sleeping on an already-cleared condition. Their TIMEDOUT arms also differ: block-completion passes dwb_set_status_resumed to dwb_remove_wait_queue_entry, the structure-mod variant passes NULL on every arm. Queue mechanics and dwb_signal_* wakers are in Chapter 10. The contract: every wait returns NO_ERROR (retry), ER_CSS_PTHREAD_COND_TIMEDOUT (retry), or a hard error (propagate); goto start re-reads the cursor on every retry.
VPID_SET (&dwb_slot->vpid, io_page_p->prv.volid, io_page_p->prv.pageid);/* <- set LAST: now identifiable */
dwb_slot->ensure_metadata= ensure_metadata;
The branch turns on prv.pageid: a real page (!= NULL_PAGEID) is memcpy’d whole into the slot’s io_page; a hole (== NULL_PAGEID) gets fileio_initialize_res for a well-formed image. Then lsa, vpid (set last so a half-staged slot still reads NULL-VPID), and ensure_metadata. The asserts gate integrity: prv.p_reserve_2 == 0 on the header reserved field, fileio_is_page_sane on the source page (not the copy).
Invariant 4-C — VPID is the visibility flag. A slot is valid to readers iff its vpid is non-NULL. dwb_acquire_next_slot nulls it at claim; dwb_set_slot_data re-sets it only after bytes and LSA are in place, so a reader (Chapter 8) finding a non-NULL VPID is guaranteed a fully-staged image and LSA. The write ordering alone enforces this — no per-slot lock.
A claim hands back one DWB_SLOT; only vpid is touched at acquisition.io_page, position_in_block, and block_no are pre-wired immutables (Ch 3); staging later fills io_page’s bytes plus lsa, vpid, ensure_metadata.
dwb_acquire_next_slot is a read-modify-CAS loop on start: — ATOMIC_INC_64(&pos, 0) reads, ATOMIC_CAS_64 commits, any lost race/wait/flag funnels back through goto start.
The lifecycle preamble has four arms, one proceeds: modifying-structure (wait or bail), !DWB_IS_CREATED → ER_DWB_DISABLED (partial teardown) vs. NULL-NO_ERROR (clean off), and unreachable assert(false).
The first writer sets the started bit and advances the position in one CAS word; if already set, the previous fill is unflushed and it waits via dwb_wait_for_block_completion to avoid wrap-around.
can_wait selects the failure mode:false returns NULL-NO_ERROR (Invariant 4-A); true blocks, and ER_CSS_PTHREAD_COND_TIMEDOUT is always a retry.
The claimed slot’s index is its fixed DWB-volume offset (Invariant 4-D) — blocks[block_no].slots[position_in_block], asserted on claim, never chosen by the producer.
dwb_set_slot_data needs no atomics (CAS conferred ownership); it branches on prv.pageid == NULL_PAGEID (reserved) vs. real page (memcpy), then sets LSA, VPID, ensure_metadata.
VPID is the slot’s visibility flag (Invariant 4-C) — nulled at claim, re-set last; write ordering alone hides a half-staged slot, no per-slot lock. At staging’s end the slot is reader-discoverable but not hash-indexed or flush-counted — that promotion is Chapter 5.
Chapter 4 left us with a staged slot: a (block_no, position_in_block) reservation carrying a page image plus vpid/lsa, invisible to concurrent readers. This chapter answers: how does a staged slot become discoverable, and how does writing the last slot of a block trigger a flush? Both happen inside dwb_add_page, which (1) publishes the slot through the VPID hash and (2) bumps the per-block fill counter and forks on fullness.
For why CUBRID keeps a VPID-keyed index of in-flight pages, see the high-level companion cubrid-double-write-buffer.md (“Why a hash over the staged pages” / “Decache after write”); not re-derived here.
5.1 The STAGED to HASHED transition and its invariant
A slot moves FREE -> STAGED (Chapter 4) -> {HASHED | INVALIDATED} -> FLUSHED (Chapter 6). This chapter owns the STAGED arrows: STAGED -> HASHED when dwb_slots_hash_insert returns inserted == true; STAGED -> INVALIDATED when it loses, after which the slot is later written as a no-op zero page.
Canonical invalidation primitive. Every “invalidate the slot” below means two writes: VPID_SET_NULL (&slot->vpid) so dwb_slots_hash_delete skips it (it early-returns on a null VPID, §5.5), plus fileio_initialize_res (thread_p, slot->io_page, IO_PAGESIZE) so its image becomes a zeroed page. The slot still holds its position and will be written to the DWB volume, but carries no live data and owns no hash entry. Skipping either risks a phantom hash deletion or a recovery replay of stale content (Chapter 9).
Invariant (STAGED-before-HASHED visibility). A reader can find a slot only afterdwb_slots_hash_insert has stored slots_hash_entry->slot = slot. The code publishes the pointer as the last write in the insert, after every LSA/block decision, under the entry mutex. Inverted, a reader in dwb_read_page could dereference a half-written slot and serve a torn page. The slots_hash_entry mutex makes the swap atomic against other inserters/deleters of the same VPID.
5.2 dwb_add_page — re-acquire fallback and the publish step
dwb_add_page is called with a pre-staged slot (the page-buffer flush path) or *p_dwb_slot == NULL (callers wanting it to stage one). The first branch handles the latter:
Four cases: a pre-staged slot is taken as-is; a dwb_set_data_on_next_slot error propagates; a re-acquire still yielding NULL (no DWB, or no slot even with can_wait = true) returns NO_ERROR so the caller writes the page directly; a real re-acquired slot falls through. The assert (VPID_EQ ...) enforces that the caller’s vpid and the slot’s dwb_slot->vpid (stamped by dwb_set_slot_data during staging) agree — a mismatch means the caller staged one page and is publishing another. The tighter triangle against io_page->prv is checked inside dwb_slots_hash_insert (§5.4), not here.
Branch accounting: a null VPID (e.g., a metadata-only flush marker) skips insertion but still counts toward count_wb_pages; an insert error propagates; inserted == true makes this slot the canonical owner; inserted == false (the !inserted path) means a better-or-equal slot already owns this VPID (§5.4), so this slot is invalidated per §5.1.
Every call (inserted or not) then accounts the slot against the block. The counter lives on the DWB_BLOCK struct as volatile unsigned int count_wb_pages and is the field the atomic touches:
assert_release (count_wb_pages <= DWB_BLOCK_NUM_PAGES); /* never overfill */
if (count_wb_pages < DWB_BLOCK_NUM_PAGES)
needs_flush =false;
else
needs_flush =true;
if (needs_flush ==false)
return NO_ERROR;
ATOMIC_INC_32 serializes the “who wrote the last slot” decision, so exactly one caller observes count_wb_pages == DWB_BLOCK_NUM_PAGES and takes the flush fork. assert_release (not stripped in production) guards the bound — firing it would mean Chapter 4 over-handed positions. Invalidated !inserted slots still increment the counter (they hold a real position). The common needs_flush == false case returns immediately.
In SERVER_MODE with a daemon available, wake the daemon and return immediately — no flush latency, no wait on the previous block (ordering is the daemon’s job, Chapter 7). Standalone, or no daemon, the caller flushes inline via dwb_flush_block (thread_p, block, false, NULL): the false is the file_sync_helper_can_flush parameter and NULL is current_position_with_flags (verified against the prototype). dwb_flush_block itself decides what remove_from_hash to pass to dwb_write_block, which runs the hash-delete loop (§5.5). Chapter 6 dissects it.
5.4 dwb_slots_hash_insert — find-or-insert and the LSA arms
find_or_insert writes true if it created a fresh entry (VPID absent), false if one existed — either way returning slots_hash_entrymutex-held, so this function must unlock on every exit. The two asserts verify the VPID triangle: hash key, slot vpid, and io_page->prv all agree.
flowchart TD
A[find_or_insert, mutex held] --> B{inserted == true?}
B -- yes, fresh --> Z[slot is owner]
B -- no, existed --> C{LSA_LT?}
C -- yes, slot older --> D[unlock, return, inserted stays false]
C -- no --> E{LSA_EQ?}
E -- yes --> F{same block_no?}
F -- yes --> G[invalidate OLD slot per 5.1]
F -- no --> H[debug-only assert old block flushes first]
E -- no, slot newer --> I[no invalidation]
G --> J[OR-merge ensure_metadata]
H --> J
I --> J
J --> Z
Z --> K[slot pointer swap, unlock, inserted = true]
Figure 5-1: dwb_slots_hash_insert decision tree.
The fresh-insert arm (*inserted == true) falls straight to the swap. The collision arm (*inserted == false) has a live owner and runs three LSA sub-arms (Fig. 5-1):
A smaller LSA means an older version, so LSA_LT (this slot is older) leaves the existing owner and early-returns inserted == false — §5.2 then invalidates this slot. LSA_EQ is the “flushing without logging” case (two slots legitimately share an LSA, latest wins): a same-block loser is invalidated in place per §5.1, targeting the existing owner, so the block holds no duplicate VPID at flush time. A different-block loser needs no invalidation (it flushes independently); the #if !defined (NDEBUG) block compiles only in debug builds, reads the old owner’s block_no with ATOMIC_INC_32 (..., 0) (a racing flush may zero it), and — only if that read is > 0 — asserts the older page’s block precedes the new one in flush order: assert ((old_block->version < new_block->version) || (old_block->version == new_block->version && old_block->block_no < new_block->block_no)), so cross-block crash recovery stays ordered (Chapter 9). The implicit-else newer-LSA arm needs no invalidation — the old owner is unhooked by the swap and dwb_slots_hash_delete no longer matches it (§5.5). The ensure_metadata OR-merge runs on every collision sub-arm so a metadata-sync request against the replaced version is never dropped.
The pointer swap is reached by the fresh-insert and LSA_EQ/newer-LSA arms, but not the LSA_LT early return:
This store is the STAGED -> HASHED transition (§5.1) — the last write before the mutex releases, satisfying the visibility invariant. The final *inserted = true normalizes both “created fresh” and “replaced an owner”: in both, this slot owns the VPID and must not be invalidated. Only LSA_LT returns false.
5.5 dwb_slots_hash_delete — the decache after write
The reverse operation, called per non-null slot from dwb_write_block’s if (remove_from_hash) loop after a block’s pages reach their home volumes. It removes the slot so future reads go to the page buffer / home volume, not the DWB copy.
Branch accounting: a VPID_ISNULL slot was invalidated (§5.1) and never hashed — exactly why invalidation nulls the VPID. A NULLfind means the entry is already gone (no-op; find returns mutex-held when it does find one). slot == slot erases via erase_locked (consuming the mutex); a failure while holding it is should-never-happen, flagged by assert_release (false) and a dwb_log_error — note the return error_code there still carries NO_ERROR (the local is never reassigned), so even in production this stays a silent no-op rather than a propagated failure. The else arm means a newer slot replaced this one after hashing; this stale slot must not erase the entry or it tears down the newer owner.
Invariant (only the current owner decaches). A slot deletes its entry only if slots_hash_entry->slot == slot. With the §5.4 swap, a VPID’s entry has one well-defined owner at every instant.
The hashed object is DWB_SLOTS_HASH_ENTRY. Beyond the payload (vpid, slot pointer, mutex), three fields are lockfree-hashmap plumbing wired by slots_entry_Descriptor (an LF_ENTRY_DESCRIPTOR):
Field
Role
vpid
The hash key (one VPID per entry).
slot
The owning DWB_SLOT * — NULL until the first insert; the §5.4 swap target.
mutex
Per-entry lock; held across find/insert/delete of one VPID.
stack, next
Freelist / hash-chain links for cubthread::lockfree_hashmap.
del_id
Delete transaction id used by the lock-free reclamation.
Six callbacks are registered in slots_entry_Descriptor:
Callback
What it does
dwb_slots_hash_entry_alloc
malloc + pthread_mutex_init; NULL/ER_OUT_OF_VIRTUAL_MEMORY on failure.
dwb_slots_hash_entry_free
pthread_mutex_destroy + free; ER_FAILED on NULL.
dwb_slots_hash_entry_init
VPID_SET_NULL (&vpid) and slot = NULL — the empty state before find_or_insert.
dwb_slots_hash_key_copy
VPID_COPY (dest, src).
dwb_slots_hash_compare_key
0 iff VPID_EQ, else -1.
dwb_slots_hash_key
(pageid | (volid << 24)) % hash_table_size — folds volid into high bits to spread volumes across buckets.
The descriptor’s duplicate / insert callback slots are NULL (“no inserts”). Population order: key_copy (during find_or_insert) sets vpid; dwb_slots_hash_insert sets slot.
5.7 dwb_get_next_block_for_flush — the fill-order gate
*block_no = DWB_NUM_TOTAL_BLOCKS; /* <- sentinel: "nothing to flush" */
if (dwb_Global.blocks[dwb_Global.next_block_to_flush].count_wb_pages!= DWB_BLOCK_NUM_PAGES)
return; /* <- next block not full yet */
*block_no =dwb_Global.next_block_to_flush;
Two branches: if next_block_to_flush is not full, return the sentinel DWB_NUM_TOTAL_BLOCKS — a full later block still cannot flush ahead of it (that would break crash-recovery ordering, Chapter 9). If full, hand back its index; the daemon (Chapter 7) flushes it and advances next_block_to_flush. This is why §5.3’s wakeup needs no wait on the previous block: the daemon re-asks here and only gets the next-in-order full block. The DWB_BLOCK_NUM_PAGES test matches dwb_add_page’s, so the two views of “full” never diverge.
dwb_add_page publishes and accounts. It re-acquires a slot when *p_dwb_slot == NULL (returning NO_ERROR with a null slot if the DWB is unavailable), publishes via dwb_slots_hash_insert, then atomically bumps count_wb_pages and forks on fullness.
Invalidation is one primitive: VPID_SET_NULL plus fileio_initialize_res (§5.1) — nulling the VPID makes decache skip the slot, re-initializing io_page makes it benign on replay. The !inserted path and the LSA_EQ same-block arm both apply it.
dwb_slots_hash_insert resolves collisions by LSA.LSA_LT keeps the newer owner and early-returns inserted == false; LSA_EQ (modified-without-logging) replaces and, if same-block, invalidates the old slot, while a different-block collision only runs a debug-build (#if !defined (NDEBUG)) ordering assert; a newer LSA replaces. The winner always reports inserted == true.
The pointer swap is the STAGED -> HASHED visibility point — slots_hash_entry->slot = slot is the last write under the entry mutex, so readers never see a half-published slot; ensure_metadata is OR-merged across replacements.
dwb_slots_hash_delete erases only when slots_hash_entry->slot == slot — a null-VPID slot, a missing entry, or one owned by a newer slot are all no-ops.
The flush fork is mode-dependent. SERVER_MODE with a daemon wakes it without flush latency; otherwise flush inline via dwb_flush_block (..., file_sync_helper_can_flush = false, NULL).
dwb_get_next_block_for_flush enforces fill order — it releases next_block_to_flush only when that exact block is full, else returns the DWB_NUM_TOTAL_BLOCKS sentinel.
Chapter 6: Flushing a Block to the DWB Volume and Home
The reader question: when a full block is flushed, what is the exact write-and-fsync sequence that makes the page durable in the DWB before it is written home, so a torn home page is always repairable?
The companion (cubrid-double-write-buffer.md) explains why the page must reach the DWB durably before home. This traces the how — dwb_flush_block, dwb_write_block, and five helpers — branch by branch. The producer that hands a full block to the flush daemon is the page-buffer flush path; that machinery is not re-derived here — see cubrid-page-buffer-manager-detail.md. The two write primitives fileio_write / fileio_write_pages and the fileio_synchronize fsync wrapper live in the disk manager; see cubrid-disk-manager-detail.md for their internals (retry, partial-write handling, O_DIRECT vs buffered). This chapter treats them as the atoms “write these pages” and “fsync this descriptor.” No new struct: it reuses DWB_BLOCK (Ch 1), DWB_SLOT (Ch 1, 4), and introduces FLUSH_VOLUME_INFO.
6.1 The flush-time volume record: FLUSH_VOLUME_INFO
A block’s pages can span several data volumes. The flusher must fsync each distinct volume exactly once after writing its pages home, and must be able to hand that fsync to a helper without two threads syncing the same volume. FLUSH_VOLUME_INFO makes that coordination race-free.
The fsync target. Set by dwb_add_volume_to_block_flush_area from the vol_fd argument, which dwb_write_block obtained upstream via fileio_get_volume_descriptor.
num_pages
Pages written home but not yet fsynced
Lets the flusher skip a volume drained to 0 by the helper, and lets max_pages_to_sync route big volumes to the helper. volatile: flusher writes, helper reads.
all_pages_written
True once every page of this volume is written
Producer-to-consumer gate: the helper must not fsync until the flusher promises no more pages are coming.
metadata
ensure_metadata argument forwarded to fileio_synchronize
A grown volume needs inode/size metadata synced too. ORed across all slots touching the volume.
flushed_status
Which thread has claimed this volume’s fsync
Three-state enum (VOLUME_NOT_FLUSHED / VOLUME_FLUSHED_BY_DWB_FILE_SYNC_HELPER_THREAD / VOLUME_FLUSHED_BY_DWB_FLUSH_THREAD); the CAS arbiter for exactly-once fsync. Flusher and helper (Ch 7) each CAS NOT_FLUSHED to their own value; winner fsyncs, loser skips.
flowchart LR
blk["DWB_BLOCK"] -->|"flush_volumes_info[0..count]"| fvi["FLUSH_VOLUME_INFO per volume"]
fvi -->|"flushed_status CAS"| arb["flusher vs helper arbiter"]
fvi -->|"num_pages / all_pages_written"| handoff["producer to consumer gate"]
Figure 6-1. A block owns an array of per-volume flush records; each arbitrates one volume’s fsync.
Invariant (exactly-once fsync per volume). At most one of {flusher, helper} runs fileio_synchronize on each record, enforced by ATOMIC_CAS_32 on flushed_status from VOLUME_NOT_FLUSHED. If violated, two threads fsync the same descriptor and num_pages (zeroed by the winner via ATOMIC_TAS_32) double-counts. The CAS makes the num_pages == 0 -> skip shortcut sound. (The helper side of this same CAS is detailed in Ch 7.)
6.2 Building the ordered snapshot: dwb_block_create_ordered_slots and friends
dwb_flush_block first builds a VPID-sorted private copy of the slot array so (a) all pages of one volume are contiguous (one descriptor lookup, one fsync) and (b) duplicate VPIDs sit adjacent so the older copy can be dropped.
*p_ordered_slots_length = block->count_wb_pages +1; /* length INCLUDES the sentinel */
Two branches: malloc failure returns OOM (the caller dwb_flush_block collapses it to ER_FAILED and goto end); otherwise the copy is sorted and returned. qsort sorts only the first count_wb_pages entries, leaving the sentinel pinned high. dwb_init_slot makes that slot a sentinel — NULL VPID, LSA, and io_page (slot->io_page = NULL; VPID_SET_NULL (&slot->vpid); LSA_SET_NULL (&slot->lsa);) — so the [i + 1] probe in dwb_write_block runs without a separate bounds check.
dwb_compare_slots sorts lexicographically over (volid, pageid, lsa.pageid, lsa.offset). The LSA tiebreak is load-bearing: two slots with the same VPID end up adjacent with the older (smaller LSA) copy at the lower index, which the dedup loop relies on to keep the newer copy.
Invariant (sort total order with sentinel). Indices 0..count_wb_pages-1 are non-decreasing under dwb_compare_slots; index count_wb_pages is a NULL sentinel (enforced by qsort + dwb_init_slot). A non-NULL sentinel would make dwb_write_block’s assert (VPID_ISNULL (&p[i+1].vpid) || VPID_LT (...)) fire on the final real page.
Branch-complete numbered walkthrough (S0–S8). There are fourgoto end sites — (1) ordered-snapshot build OOM (S1), (2) DWB-volume fileio_write_pages returns NULL (S5), (3) DWB-volume fileio_synchronize mismatch (S5), (4) dwb_write_block returns non-NO_ERROR (S6) — and every one of them, plus normal fall-through, lands at end:. The durability barrier is S5.
S0 asserts single-flusher (ATOMIC_INC_32 (&dwb_Global.blocks_flush_counter, 1); assert (dwb_Global.blocks_flush_counter <= 1);); the matching -1 at end: is hit by every exit including all goto end branches, which is why error paths jump rather than return. S1 builds the ordered snapshot (6.2); on failure error_code = ER_FAILED; goto end. S2 is the dedup loop below, over i = 0 .. count_wb_pages - 2 (for (i = 0; i < count_wb_pages - 1; i++)), so s2 = &p[i+1] reaches at most the last real slot — the loop never touches the sentinel. S3 (SERVER_MODE) spins while file_sync_helper_block != NULL to ensure the previous block is durable before this one’s home writes begin: helper alive -> thread_sleep(1) and respin; helper gone -> the flusher fsyncs the previous block’s volumes itself (the inner if (ATOMIC_INC_32(&num_pages, 0) >= 0) guard is effectively always-true, so every volume is synced) and ATOMIC_TAS_ADDRs file_sync_helper_block to NULL, breaking the loop. S4 resets ATOMIC_TAS_32 (&count_flush_volumes_info, 0) and all_pages_written = false.
The dedup branch fires only on two adjacent same-VPID slots. It does three things: nulls the older copy in both snapshot and live block (so the home write in dwb_write_block skips it and the hash-delete pass leaves the survivor alone), reinitializes the dropped slot’s io_page with fileio_initialize_res (because S5 writes the wholewrite_buffer to the DWB volume — the dropped page must become a clean reinitialized page, not stale bytes), and carries ensure_metadata forward to the survivor. A #if !defined(NDEBUG) WAL check (logpb_need_wal against the page LSA) sits in the same loop body but is a debug-only probe, no runtime branch.
The whole write_buffer (including the just-reinitialized deduped pages, written as harmless clean pages) goes to the single DWB volume in onefileio_write_pages (cubrid-disk-manager-detail.md for how that call retries/handles partial writes); the write returning NULL and the sync returning anything but dwb_Global.vdes are the two goto end branches. The DWB-volume fsync passes ensure_metadata = false because the DWB volume is fixed-size and pre-created — only home volumes can grow. S6 then calls dwb_write_block (thread_p, block, p_dwb_ordered_slots, ordered_slots_length, file_sync_helper_can_flush, /*remove_from_hash=*/true) (6.4); on error, assert (false); goto end.
Invariant (durability barrier — the whole point of a double-write buffer).fileio_synchronize of dwb_Global.vdes MUST complete before the first home write begins — enforced by straight-line sequencing (dwb_write_block is the very next statement, and the sync’s error branch goto end skips it). If violated, a crash mid-home-write could leave a home page torn AND the DWB copy not yet durable, so recovery (Ch 9) would have a corrupt home and no good DWB copy to restore from. Because the barrier holds, at crash time every home page that is being (or about to be) written has a complete, fsync’d twin in the DWB volume; Ch 9’s restart scan copies those twins back over any torn home, making torn-page repair always possible.
S7 — per-volume fsync of home volumes (fvi abbreviates block->flush_volumes_info below):
The three continues (a)/(b)/(c) are annotated inline. (a) and (c) are both reached when the helper got there first — (a) sees the drained counter, (c) loses the status CAS. The (b’) else-branch (when file_sync_helper_can_flush is false, e.g. SA_MODE or no daemon) asserts no helper exists, so every surviving volume is fsynced inline. A volume the flusher claims ATOMIC_TAS_32-zeroes num_pages before fsyncing so the helper’s num_pages == 0 check skips it.
S8 — publish completion in fixed order: all_pages_written = true (S7-end, so the helper may finish) -> ATOMIC_TAS_32 (&count_wb_pages, 0) -> ATOMIC_INC_64 (&version, 1ULL) -> clear the WRITE-STARTED bit -> advance next_block_to_flush -> dwb_signal_block_completion -> store *current_position_with_flags. The two CAS sites differ in retry policy: clearing the WRITE-STARTED bit in the shared position word (Ch 2, via DWB_ENDS_BLOCK_WRITING) is a reset_bit_position: retry loop because a concurrent producer may touch the word; advancing next_block_to_flush is a CAS whose failure branch fires assert_release (false) — it must win, since only the single flusher advances it. dwb_signal_block_completion destroys the wait queue and wakes blocked threads (Ch 8); the version++ tells stale readers (Ch 8) their block was recycled.
S end — cleanup. Reached by fall-through and every goto end: ATOMIC_INC_32 (&blocks_flush_counter, -1), free_and_init the snapshot if allocated, charge PSTAT_DWB_FLUSH_BLOCK_TIME_COUNTERS, return error_code.
dwb_write_block walks the snapshot, writes each live page home, builds flush_volumes_info[] at volume boundaries, and opportunistically hands volumes to the helper. The per-page loop (every branch covered):
NULL VPID (deduped slot or the trailing sentinel): continue — not written, not counted.
New volid: close the previous record (all_pages_written = true, raise can_flush_volume, reset current_flush_volume_info = NULL), then fileio_get_volume_descriptor. A dropped volume (NULL_VOLDES) continues without opening a record — page silently skipped. Otherwise dwb_add_volume_to_block_flush_area opens the new record.
Same volid: current_flush_volume_info->metadata = metadata || ensure_metadata.
fileio_write HOME with NO_COMPENSATE_WRITE (the disk-manager primitive; cubrid-disk-manager-detail.md); the lone NULL does return ER_FAILED directly (not goto end) — dwb_flush_block turns that into its own goto end.
Handoff (server mode), the one concurrency detail:
ATOMIC_INC_32 (¤t_flush_volume_info->num_pages, 1); /* count this home write; count_writes++ too */
if (file_sync_helper_can_flush && (count_writes >= num_pages_to_sync || can_flush_volume ==true)
&&dwb_is_file_sync_helper_daemon_available ())
{
if (ATOMIC_CAS_ADDR (&dwb_Global.file_sync_helper_block, (DWB_BLOCK *) NULL, block)) /* claim only if NULL */
dwb_file_sync_helper_daemon->wakeup ();
/* then reset count_writes = 0, can_flush_volume = false */
}
Three tail actions follow the loop: close the last opened volume (all_pages_written = true); a second helper-wakeup for small blocks whose count_writes never hit the threshold (guarded by file_sync_helper_block == NULL && count_flush_volumes_info > 0); and, when remove_from_hash is set, the hash-delete pass (6.5).
Invariant (volume record precedes its pages’ counting).dwb_add_volume_to_block_flush_area publishes the record (ATOMIC_INC_32 on count_flush_volumes_info) before any num_pages increment for that volume — enforced because the add returns the record pointer used for the first fileio_write/num_pages++. Otherwise the helper could read num_pages on a record not yet inside the array bound and miss pages.
When remove_from_hash is true (it always is, from dwb_flush_block’s call), dwb_write_block’s tail removes each surviving slot from the global VPID->slot hash so future producers stop seeing the flushed pages. The loop calls dwb_slots_hash_delete on the liveblock->slots[p_dwb_ordered_slots[i].position_in_block] (not the snapshot copy) so the slot == slot identity check below compares the exact address the hash stored:
pthread_mutex_unlock (&slots_hash_entry->mutex); /* a newer slot owns this VPID now: just unlock */
Three branches (inline-annotated): NULL VPID skip; no entry skip; entry found — erase_locked only if slots_hash_entry->slot == slot (still this exact slot), else the hash now maps the VPID to a different slot (a newer producer re-bound it to a fresher copy) so we just unlock without erasing. That else stops the flusher from deleting an entry a concurrent writer just re-bound. A source-side guard asserts the found entry’s VPID is either NULL or equal to ours (VPID_ISNULL (&entry->slot->vpid) || VPID_EQ (...)) before the identity test. find returns with the entry mutex held; both the erase_locked path and the else release it (erase_locked internally; the else via explicit pthread_mutex_unlock).
6.6 dwb_add_volume_to_block_flush_area and dwb_compare_vol_fd
ATOMIC_INC_32 (&block->count_flush_volumes_info, 1); /* publish AFTER all fields are set */
return flush_new_volume_info;
Every field is set, then the count is bumped with an atomic increment — the source comment notes the atomic exists “to prevent code reordering” (one writer, two readers: flusher and helper) so daemons never see a half-built element. A trailing assert confirms the count grew by exactly one and stays within max_to_flush_vdes (array sized at block creation, Ch 3).
dwb_compare_vol_fd (return ((*vol_fd1) - (*vol_fd2));) is a plain int-descriptor qsort comparator. It is not invoked in the flush path (and has no caller in double_write_buffer.cpp as of this revision); it is retained as a descriptor-array comparator and is listed here only because it lives in the same flush-area bookkeeping family. Do not infer a call from dwb_flush_block or dwb_write_block.
The durability barrier is one line, and it is why a DWB exists. The whole write_buffer goes to the single DWB volume in one fileio_write_pages (NO_COMPENSATE_WRITE); the immediately following fileio_synchronize (dwb_Global.vdes, ...) is sequenced strictly before dwb_write_block scatters pages home, and its error branch goto end skips the home write. The DWB copy is fsync-durable first, so any home page later found torn after a crash has a clean twin to restore from (Ch 9).
The ordered snapshot does double duty. Sorting by (volid, pageid, lsa) makes volumes contiguous (one fsync each) and same-VPID duplicates adjacent; the dedup loop keeps the higher-LSA copy, nulls the older in both snapshot and live block, and fileio_initialize_res-reinitializes the dropped page so the DWB write emits a clean page. A NULL sentinel at count_wb_pages makes the [i+1] probe bounds-safe.
fsync is exactly-once per volume via a 3-state CAS.flushed_status + ATOMIC_CAS_32 arbitrates flusher vs helper; num_pages == 0 and the max_pages_to_sync threshold route large volumes to the helper, the CAS loser skips. Records publish only after every field is set; the hash-delete pass (in dwb_write_block, not dwb_flush_block) erases only if the entry still points at this live slot.
Completion is published in a fixed order.all_pages_written -> count_wb_pages = 0 -> version++ -> clear WRITE bit (CAS-retry loop) -> advance next_block_to_flush (CAS whose failure branch is assert_release (false)) -> dwb_signal_block_completion. The version bump tells stale readers (Ch 8) their block was recycled.
The two write primitives and the flush producer are borrowed, not re-derived.fileio_write / fileio_write_pages / fileio_synchronize belong to the disk manager (cubrid-disk-manager-detail.md); the page-buffer flush path that fills and hands off a block is in cubrid-page-buffer-manager-detail.md. The helper thread that shares the per-volume fsync CAS is Ch 7.
Chapter 7: Daemons Flush Driver and File Sync Helper
The DWB drains itself through two background threads. The flush driver, registered as "dwb-flush-block" (1 ms looper), picks a full block and runs the whole write-and-sync pipeline; the helper, registered as "dwb-file-sync" (10 ms looper), offloads the slowest part — the per-home-volume fileio_synchronize calls — so the flusher does not block on disk while later writers keep filling the DWB volume. The handoff is a single pointer (dwb_Global.file_sync_helper_block) plus a per-volume atomic status field. This chapter dissects that handoff, the daemon bodies, and the availability predicates.
Cross-link: double-buffering and torn-page theory are in the high-level companion cubrid-double-write-buffer.md; block fill and the dwb_flush_block body are Chapters 5 and 6.
Both daemons are module-global cubthread::daemon * pointers (dwb_flush_block_daemon, dwb_file_sync_helper_daemon), defined only under #if defined (SERVER_MODE) and NULL until created. Every path touching them is gated by PRM_ID_ENABLE_DWB_FLUSH_THREAD; when off, the DWB degrades to inline flush on the writer’s thread. A second gate inside each body, BO_IS_FLUSH_DAEMON_AVAILABLE(), reads boot_Enabled_flush_daemons (cleared during boot/restart). Both must hold: server-past-boot and operator opt-in.
Figure 7-1 shows the relationship around the handoff slot.
flowchart LR
FB["dwb-flush-block daemon<br/>looper 1 ms"]
FH["dwb-file-sync daemon<br/>looper 10 ms"]
HP["file_sync_helper_block<br/>single DWB_BLOCK* slot"]
HOME["home volumes<br/>fileio_synchronize"]
FB -->|"CAS NULL to block + wakeup"| HP
HP -->|"dwb_file_sync_helper reads it"| FH
FH --> HOME
FB -->|"fallback: inline fsync if helper unavailable"| HOME
Figure 7-1. The handoff slot. Note the registered daemon names are "dwb-flush-block" and "dwb-file-sync" (the latter is not"dwb-file-sync-helper", despite the function-name prefix dwb_file_sync_helper_*).
dwb_daemons_init runs once during DWB creation, constructing both daemons in order: dwb_flush_block_daemon_init() then dwb_file_sync_helper_daemon_init(). Each builds a cubthread::looper (wake cadence) and a task object, then calls cubthread::get_manager()->create_daemon(looper, task, name). They differ in cadence and wrapper:
dwb_flush_block_daemon_init: looper(milliseconds(1)) + a bespoke dwb_flush_block_daemon_task, named "dwb-flush-block".
dwb_file_sync_helper_daemon_init: looper(milliseconds(10)) + a generic entry_callable_task(dwb_file_sync_helper_execute), named "dwb-file-sync".
Each init function is preceded by a REGISTER_DAEMON(...) macro at file scope that statically registers the daemon name for the thread manager (REGISTER_DAEMON (dwb_flush_block), REGISTER_DAEMON (dwb_file_sync_helper)).
The asymmetry is deliberate: the flusher’s dedicated task class carries a PERF_UTIME_TRACKER for cond-wait time (Section 7.3), while the stateless helper needs only the generic wrapper. The cadence gap matters — the flusher must react quickly when a block fills (else writers stall on a free slot), whereas the helper only keeps up with the volumes handed to it.
dwb_daemons_destroy tears both down (flusher first) via destroy_daemon(...). destroy_daemon joins the thread, so on return neither body can be mid-flight. The pointers stay non-NULL here; teardown (Chapter 10) ensures no producer dereferences a destroyed daemon.
dwb_flush_block_daemon_task derives from cubthread::entry_task and holds one member, PERF_UTIME_TRACKER m_perf_track, started in its constructor. The execute override is the per-tick entry — a thin wrapper respecting both gates:
The PERF_UTIME_TRACKER pair brackets the work so the PSTAT_DWB_FLUSH_BLOCK_COND_WAIT counter accumulates only idle time: _TIME stops the clock at the start of an active tick, _START restarts it as the body returns to sleep. The assert_release(false) is load-bearing: the flush daemon has no recovery path — a flush error aborts the server rather than leaving a block half-flushed.
dwb_flush_next_block is the real driver — a goto start-driven loop draining every ready block. Each iteration re-reads position_with_flags = ATOMIC_INC_64 (&dwb_Global.position_with_flags, 0ULL) and branches: (1) if !DWB_IS_CREATED || DWB_IS_MODIFYING_STRUCTURE, return NO_ERROR; (2) if dwb_get_next_block_for_flush yields block_no < DWB_NUM_TOTAL_BLOCKS, call dwb_flush_block (..., true /* file_sync_helper_can_flush */, NULL); (3) on its error, return error_code (caller turns it into assert_release(false)); (4) on success, goto start since another block may now be full; (5) no full block — return NO_ERROR, done for this tick.
The position_with_flags re-read (Chapter 2) makes the loop teardown-safe — a structure modification flips MODIFYING, so the next iteration bails at branch 1. Branch 2 vs 5 is decided by dwb_get_next_block_for_flush, which first sets the sentinel *block_no = DWB_NUM_TOTAL_BLOCKS and overwrites it with dwb_Global.next_block_to_flush only when dwb_Global.blocks[next_block_to_flush].count_wb_pages == DWB_BLOCK_NUM_PAGES — it offers only next_block_to_flush, and only when completely full.
Invariant — blocks flush in fill order. The daemon never picks an arbitrary full block, only next_block_to_flush; the cursor advances round-robin inside dwb_flush_block (the ATOMIC_CAS_32 on dwb_Global.next_block_to_flush, near the end of the body), so on-disk write order matches buffering order. Crash-recovery replay (Chapter 9) depends on this — an out-of-order flush could land a newer page version before an older one and recovery would resurrect stale data. The count_wb_pages guard also ensures the block is handed over only when every slot is written. The call passes file_sync_helper_can_flush = true — the green light to delegate home-volume fsyncs to the helper (Chapter 6 covers the body; Section 7.4 the receiving end).
The per-tick wrapper dwb_file_sync_helper_execute mirrors execute’s two gates — return if !BO_IS_FLUSH_DAEMON_AVAILABLE(), then call dwb_file_sync_helper(&thread_ref) only if PRM_ID_ENABLE_DWB_FLUSH_THREAD — but with no error escalation: it ignores the return value (best-effort; any missed volume is synced inline by the flusher, Section 7.5). dwb_file_sync_helper reads the handoff pointer once, then walks that block’s flush_volumes_info array (Figure 7-2).
flowchart TD
B{"created and not modifying?"}
B -- no --> Z1["return NO_ERROR"]
B -- yes --> D{"file_sync_helper_block is NULL?"}
D -- yes --> Z1
D -- no --> E["snapshot count_flush_volumes_info and all_pages_written"]
E --> F["for i = start_flush_volume .. count"]
F --> G{"already FILE_SYNC_HELPER?"}
G -- no --> H{"CAS NOT_FLUSHED to FILE_SYNC_HELPER"}
H -- fail --> F
H -- ok --> J["own volume"]
G -- yes --> J
J --> K{"num_pages below num_pages_to_sync?"}
K -- "partial, more coming" --> L["need_wait, break"]
K -- "enough OR few-but-complete" --> M["TAS num_pages=0, fileio_synchronize"]
M --> F
L --> N["recompute can_flush_volume"]
F -->|loop ends| N
N --> O{"can_flush_volume?"}
O -- yes --> E
O -- "no, need_wait" --> P["thread_sleep 1, can_flush_volume=true"]
P --> E
O -- "no, done" --> Q["TAS_ADDR file_sync_helper_block=NULL, return"]
Figure 7-2. The helper’s outer do-while over volumes.
After the two early returns (branch 1 !DWB_IS_CREATED || DWB_IS_MODIFYING_STRUCTURE, branch 2 block == NULL), the body is an outer do { ... } while (can_flush_volume == true) wrapping an inner for over the volumes. Retry subtlety: when need_wait is set and no volume could be flushed, the helper does thread_sleep(1) then sets can_flush_volume = true, re-entering the do-while to pick up data that arrived during the sleep (Figure 7-2 edge P --> E).
Invariant — exactly one thread fsyncs each volume.flushed_status is the three-state enum FLUSH_VOLUME_STATUS (VOLUME_NOT_FLUSHED, VOLUME_FLUSHED_BY_DWB_FILE_SYNC_HELPER_THREAD, VOLUME_FLUSHED_BY_DWB_FLUSH_THREAD), a member of flush_volume_info alongside vdes, the atomic num_pages, all_pages_written, and metadata. The helper claims via CAS NOT_FLUSHED → FILE_SYNC_HELPER; the flusher’s inline path (Section 7.5, Chapter 6) does the symmetric CAS to FLUSH_THREAD. The winner owns the fileio_synchronize, the loser continues. Break it and the volume is fsynced twice, or — without atomicity — skipped by both (a recovery hole). The post-fail assert_release pins the only legal loser-state: FLUSH_THREAD. The assert_release(num_pages2 >= num_pages) after the TAS pins the companion invariant: between the two reads a concurrent writer can only add pages, never remove them.
The gating below the claim (the J → K → ... nodes of Figure 7-2) balances two goals: do not fsync a volume holding a handful of dirty pages while the writer still streams into it (break with need_wait), but do not wait forever once all_pages_written is set (num_pages_to_sync = PRM_ID_PB_SYNC_ON_NFLUSH). The ATOMIC_TAS_32(..., 0) coordinates counts: reset to 0 means “accounted for”, a concurrent writer bumps it back up, and the volume is revisited via first_partial_flushed_volume (which the loop records only on its first partial volume — note the == -1 guard, so it pins the earliest unfinished volume). After the loop, start_flush_volume advances (to first_partial_flushed_volume if any, else past the last volume), and can_flush_volume/need_wait are recomputed against the possibly-grown count_flush_volumes_info; on the final pass the release ATOMIC_TAS_ADDRs the slot back to NULL (guarded by assert (block == dwb_Global.file_sync_helper_block)).
Invariant — the handoff slot holds at most one block.file_sync_helper_block is a single pointer, not a queue. A producer installs only via ATOMIC_CAS_ADDR(NULL, block) — never overwriting a non-NULL slot — and the helper (or inline fallback, or teardown) is the only clearer. While a block sits in the slot no other can be handed over, so the flusher’s “wait for previous block synced” loop (Section 7.5) serializes blocks — which is why they flush in fill order (Section 7.3).
7.5 Producer side: wakeup, and the inline fallback
The CAS guard enforces the single-slot invariant from the producer side: if a previous block is still parked the CAS fails and the flusher keeps the work itself. dwb_write_block installs at two sites — this mid-stream check and a tail check after the last volume — both via the same CAS. The tail check fires when file_sync_helper_can_flush && dwb_Global.file_sync_helper_block == NULL && block->count_flush_volumes_info > 0 and the helper is available, covering the case where the loop above never triggered a handoff.
When the helper is not available the flusher syncs the volumes itself. There are two distinct inline-sync sites inside dwb_flush_block: (a) before writing a new block, the while (file_sync_helper_block != NULL) drain loop — if the helper is unavailable it fsyncs the previous parked block’s volumes and ATOMIC_TAS_ADDRs the slot to NULL; (b) after dwb_write_block returns, the per-volume loop that CASes NOT_FLUSHED → FLUSH_THREAD and fsyncs each volume the helper did not claim. So producer behavior depends entirely on dwb_is_file_sync_helper_daemon_available(): available → publish and wake; unavailable → fsync inline. Correctness is identical; only latency changes. Chapter 6 dissects those loops and a third delegation point: in that post-dwb_write_block loop, a volume whose num_pages > PRM_ID_PB_SYNC_ON_NFLUSH / 2 is left for the helper when available (and asserts the slot is already non-NULL).
Note the producer-to-flusher hand-off is itself a wakeup, not a call: when a block fills, the writer thread (in the path leading to dwb_acquire_next_slot’s block-full branch) wakes dwb_flush_block_daemon if dwb_is_flush_block_daemon_available(), then returns; the comment there records that the explicit “wait for previous block” was removed because the flush daemon already enforces fill order. Only if the flush daemon is unavailable does the writer call dwb_flush_block(..., false, NULL) inline on its own thread.
Four boolean helpers share one shape under #if defined (SERVER_MODE) and return false otherwise (why standalone always flushes inline). The available pair (“should I delegate work?”) returns PRM_ID_ENABLE_DWB_FLUSH_THREAD == true && dwb_flush_block_daemon != NULL; the running pair appends && dwb_flush_block_daemon->is_running () — “is the daemon currently inside its body?”. The file-sync-helper variants are identical against dwb_file_sync_helper_daemon.
The distinction is load-bearing for structure modification (Chapter 10): a modifier sets MODIFYING, then spins with thread_sleep(20) while ATOMIC_INC_32 (&dwb_Global.blocks_flush_counter, 0) > 0 || dwb_flush_block_daemon_is_running() || dwb_file_sync_helper_daemon_is_running(). *_available is not enough — a daemon can be available yet mid-flush; the modifier must wait for running to clear to become sole accessor. The daemons’ per-tick position_with_flags re-read makes them bail on MODIFYING, so the loop exits.
Two daemons split one pipeline."dwb-flush-block" (1 ms) runs the full write-and-sync for one block; "dwb-file-sync" (10 ms) offloads the per-home-volume fileio_synchronize calls. The helper’s function prefix is dwb_file_sync_helper_*, but its registered daemon name is "dwb-file-sync".
The handoff is a single pointer.dwb_Global.file_sync_helper_block holds at most one block; producers install via ATOMIC_CAS_ADDR(NULL → block), the helper clears via ATOMIC_TAS_ADDR(→ NULL). The single slot serializes blocks.
Per-volume ownership is a CAS on a 3-state enum.flushed_status (FLUSH_VOLUME_STATUS, a field of flush_volume_info) goes NOT_FLUSHED → FILE_SYNC_HELPER or → FLUSH_THREAD; the CAS guarantees exactly one thread fsyncs each home volume, with assert_release pinning both the loser-state and the num_pages2 >= num_pages monotonicity.
Blocks flush strictly in fill order.dwb_get_next_block_for_flush offers only next_block_to_flush, and only when completely full; the cursor advances via a CAS inside dwb_flush_block — a recovery-correctness invariant.
The daemon path is best-effort with an inline fallback.dwb_is_*_daemon_available() decides delegate vs. sync-on-own-thread; unavailable means dwb_flush_block fsyncs the parked volumes inline (two sites: the pre-write drain loop and the post-write per-volume loop). Correctness is identical; only latency changes.
Two gates plus a boot flag protect every path.PRM_ID_ENABLE_DWB_FLUSH_THREAD (opt-in) and BO_IS_FLUSH_DAEMON_AVAILABLE() must both hold; standalone builds short-circuit *_available to false.
available ≠ running. Structure modification waits on dwb_*_daemon_is_running() — pointer set andis_running() — so the modifier becomes the sole accessor first.
Chapter 8: Reader Hits Producers Outside Page Buffer and Force Drain
Chapters 3–7 traced the DWB from the inside; this chapter covers its
external surface — three ways the engine touches a live DWB: the read
side (a page-fix miss may find a fresher copy in a slot), non-pgbuf
producers (file_io.c direct writes that must funnel through the DWB), and
force drain. For the slot-hash-as-write-back-cache overview, see the
companion cubrid-double-write-buffer.md §“Slot hash — turning the DWB into a
read-side cache”. We trace every branch of dwb_read_page,
fileio_write_or_add_to_dwb, dwb_flush_force, and dwb_synchronize, plus
the pgbuf_bcb_safe_flush_internal → pgbuf_bcb_flush_with_wal chain.
On pgbuf_fix’s on-miss path, before fileio_read, the page-buffer call site
in page_buffer.c invokes dwb_read_page inside a three-way
if/else if/else if: the first arm (non-NO_ERROR) only assert(false)
then return NULL, freeing nothing — dwb_read_page cannot in fact error, it
always returns NO_ERROR; the second arm (success == true) skips the
disk read, the page was copied straight out of a slot; the third arm runs
fileio_read, and it is that arm whose read error unwinds the BCB
(pgbuf_put_bcb_into_invalid_list + pgbuf_unlock_page). That is the whole
read contract. dwb_read_page itself is short but every branch matters:
/* assert prv.pageid/prv.volid still match vpid -- second invariant layer */
*success =true; } /* <- genuine hit */
pthread_mutex_unlock (&slots_hash_entry->mutex); } /* (D) find() left this locked */
return NO_ERROR;
(A) and (B) are straightforward (Figure 8-1); find returns the entry with
its per-entry mutex held, so (D) unlocks on both sub-branches of (C).
(C) the VPID_EQ re-check is the subtle one: between the find match and
acquiring the mutex, the entry’s slot pointer can be repointed at a slot for
a different VPID (because dwb_slots_hash_insert, Chapter 5, reuses
entries), so the guard re-validates before the memcpy. A post-memcpyassert re-confirms the copied bytes’ prv.pageid/prv.volid against the
requested VPID — a debug-build second layer on the same invariant.
INVARIANT — a DWB read hit returns a self-consistent page image. The
memcpy runs under the entry mutex and afterVPID_EQ; a producer must
take the same mutex in dwb_slots_hash_insert. Drop the re-check and a
reader could copy a slot whose bytes now belong to an unrelated page.
flowchart TB
A["dwb_read_page(vpid)"]
B{"dwb_is_created?"}
C["return: *success=false\n(caller does fileio_read)"]
D["slots_hashmap.find(vpid)\nreturns entry + holds entry mutex"]
E{"entry != NULL?"}
F{"VPID_EQ(slot->vpid, vpid)?"}
G["memcpy slot->io_page -> caller buf\n*success=true"]
H["leave *success=false\n(slot was overwritten)"]
I["pthread_mutex_unlock(entry->mutex)"]
A --> B
B -- "no" --> C
B -- "yes" --> D --> E
E -- "no (miss)" --> C
E -- "yes" --> F
F -- "yes (hit)" --> G --> I
F -- "no (raced)" --> H --> I
Figure 8-1 — dwb_read_page branch map. Only the VPID_EQ-confirmed arm
sets *success; every other path falls back to the home-volume read.
dwb_read_page finds a slot only afterdwb_add_page hashes it. The
producer stages first (dwb_set_data_on_next_slot — bytes copied in, not yet
hashed = STAGED), forces WAL, then publishes (dwb_add_page = HASHED, §8.3).
Between STAGED and HASHED, slots_hashmap.find returns NULL, so a concurrent
reader misses the DWB and reads the old home image.
INVARIANT — reading the stale home image during the STAGED window is
safe. The new image is not yet durable (WAL not even forced at the
window’s start), so the stale read returns the previous committed image —
correct. The hash is an accelerator, not a correctness gate. Peeking at a
STAGED-but-unhashed slot would expose a page whose WAL is not yet on disk.
8.3 Page-buffer producer — pgbuf_bcb_safe_flush_internal to pgbuf_bcb_flush_with_wal
The dominant producer is the page-buffer flush. pgbuf_bcb_safe_flush_internal
returns early on !pgbuf_bcb_is_dirty, then a CAS loop classifies the BCB
into three outcomes; only immediate_flush reaches the DWB:
Outcome
Condition
Action
not dirty
!pgbuf_bcb_is_dirty
return NO_ERROR — never touches the DWB
immediate_flush
not flushing AND (unlatched / read-latched / self write-latched)
pgbuf_bcb_flush_with_wal (producer body)
deferred
write-latched by another, or already flushing
set PGBUF_BCB_ASYNC_FLUSH_REQ, or park if synchronous
Inside pgbuf_bcb_flush_with_wal the DWB gate is computed once:
Two branches are non-obvious. (b) WAL FORCE runs only when
oldest_unflush_lsa is non-NULL — a NULL skips it (page dirtied without
logging, tolerated). (d) a dwb_slot == NULL from dwb_add_page (or (a))
means the DWB was torn down mid-flush → re-lock the BCB and retry the copy
down the non-DWB arm.
INVARIANT — WAL is forced between STAGE and PUBLISH, never after. The
split interposes logpb_flush_log_for_wal between
dwb_set_data_on_next_slot and dwb_add_page, so a slot becomes
reader-visible (§8.1) at PUBLISH only after its WAL is on disk — otherwise a
crash could leave the on-disk DWB copy ahead of the log.
Both producers pick the fileio_write mode by dwb_is_created(), encoding
“is the DWB the active torn-page mechanism right now”:
FILEIO_WRITE_NO_COMPENSATE_WRITE when the DWB is live (it owns torn-page
recovery, so fileio_write skips its own partial-write compensation), else
FILEIO_WRITE_DEFAULT_WRITE (fileio_write is the sole torn-page defense
and re-issues partial writes). Referenced by §8.5 (7) and summary item 5.
Direct file_io.c writes (formatting, page-array init, inter-volume copies)
funnel through fileio_write_or_add_to_dwb so a live DWB stages instead of
writing home:
Unlike the page-buffer producer it has no STAGE step (the caller owns
io_page_p, handed straight to dwb_add_page) and no WAL force (unlogged
structural writes). The inline tags carry the branches: (2) mirrors §8.3’s
is_temp gate, and (5)/(6) — DWB disabled or non-permanent — share the
direct-write fallback at (7). In CS_MODE the function is a plain
fileio_write.
A checkpoint, sync, or dismount must guarantee everything staged reaches
durable home storage first. It cannot just “flush all blocks” (producers
may be mid-fill); it picks the latest not-yet-flushed block and drives it to
completion, padding with null pages if producers stall — a goto state
machine over position_with_flags:
// dwb_flush_force -- src/storage/double_write_buffer.cpp (goto graph; tags keyed to prose)
while (dwb_Global.file_sync_helper_block ==&dwb_Global.blocks[initial_block_no])
thread_sleep (1); // (L) wait out helper fsync
end:
*all_sync =true; return NO_ERROR; // (M) caller may skip fsync
Beyond the inline tags: (E) the highest-version started block is the
freshest unflushed (older blocks are already gone by flush ordering), so
draining it suffices. (D)/(H)/(I) are tail-only — the block was flushed or
reused at a newer version, leaving only the helper’s home-fsync. (J)/(K)
padding is the forward-progress engine:
INVARIANT — dwb_flush_force makes forward progress without a live
producer. Waiting for a half-full block to fill would deadlock if the
forcer is the only DWB thread; the (J) padding makes it manufacture the
pages to reach DWB_BLOCK_NUM_PAGES and trigger the flush itself.
flowchart LR
S["Snapshot"] -- "DWB off" --> D["Done\nall_sync=true"]
S -- "helper busy" --> T["HelperTail\nwait helper"]
S -- "block started" --> P["PickBlock"] --> C{"CheckFlush"}
C -- "idle, room" --> J["Pad null page"] --> C
C -- "gone / version bump" --> T --> D
Figure 8-2 — dwb_flush_force flow; the Pad self-loop is the
forward-progress engine. (F)/(G) restart/wait arms elided.
The non-obvious branch is (1): with PRM_ID_SUPPRESS_FSYNC on,
fileio_fsync_pending returns true for all but every Nth call, so
dwb_synchronize returns untouched (an fsync-batching valve). (2) drains
only permanent volumes (complete is §8.6’s all_sync); (3) the direct
fsync runs only when complete == false, never double-fsyncing.
INVARIANT — dwb_synchronize never double-fsyncs a DWB-covered volume.
The complete == false guard at (3) runs the explicit fsync(vol_fd) only
when the drain did not already sync home volumes. Unconditional (3) would
fsync every DWB-covered volume twice per request.
The two primitives are wired into file_io.c at distinct granularities.
Note:fileio_synchronize itself does not drain the DWB — it fsyncs
directly; the drain lives in the volume-operation callers below.
Caller
Granularity
Primitive
Purpose
fileio_synchronize_all
whole database
dwb_flush_force
drain DWB, then sweep volumes only if !all_sync
fileio_reset_volume
one volume
dwb_synchronize
reset page LSAs: drain then fsync
fileio_copy_volume
one volume
dwb_synchronize
after copying via fileio_write_or_add_to_dwb, drain before declaring durable
fileio_dismount
one volume
dwb_synchronize
drain before closing the descriptor
fileio_dismount_volume
one volume
dwb_synchronize
per-volume sweep during dismount-all
fileio_synchronize_all calls dwb_flush_force (thread_p, &all_sync), then
sweeps per-volume only if all_sync == false; when true the DWB’s own
home-volume fsyncs (Chapter 6) already covered every permanent volume — the
whole-database mirror of §8.7’s complete short-circuit. The companion lists
fileio_synchronize_volume_and_dwb as a drain wrapper, but no such function
exists — the drain is inlined by the four callers above (Chapter 9 logs the
drift).
Two more drain call sites live outside file_io.c, in disk_manager.c: one
dwb_flush_force and one dwb_synchronize, both on the volume-format /
expand path. They follow the same contracts as the table above; the table is
scoped to file_io.c because that is where the bulk of volume durability is
driven.
dwb_read_page is the read-side write-back cache, consulted before
fileio_read via the three-way if/else if/else if at the call site.
The VPID_EQ re-check is the correctness guard — copy only after
confirming under the entry mutex that the reusable slot still matches.
The STAGED-not-yet-HASHED window is benign (§8.2): a miss reads the old
home image, correct because the new WAL is not yet durable.
The page-buffer producer splits STAGE / WAL-force / PUBLISH; a
dwb_slot == NULL from either DWB call triggers a non-DWB fallback.
fileio_write_or_add_to_dwb is the non-pgbuf producer (no STAGE, no
WAL force); both producers pick the write mode by §8.4.
dwb_flush_force drains by padding the latest block until it fills and
flushes, then waits out the helper’s tail.
Drain call sites avoid double-fsync via the complete/all_sync
guards and the fileio_fsync_pending throttle; fileio_synchronize itself
does not drain the DWB.
This chapter answers one question: on restart, how is the on-disk DWB
volume read back, used to repair torn home pages, and recreated fresh —
all before log redo runs? The high-level companion
(cubrid-double-write-buffer.md) explains why a torn-page window
exists; for where this fits in the broader restart pipeline see
cubrid-recovery-manager-detail.md (the pre-redo phase). Entry point:
dwb_load_and_recover_pages, invoked once by the boot sequence, with a
post-recovery guarantee:
Invariant 9-A (recovery handoff). When
dwb_load_and_recover_pages returns NO_ERROR, the old DWB volume on
disk no longer exists, a fresh empty DWB of the configured size has
been created, and every home page torn (half-written) at crash time has
been overwritten with its last fully-flushed, checksum-valid DWB image.
Log redo therefore replays onto coherent (non-torn) pages.
The ordering is load-bearing in three ways: beforedwb_daemons_init
(recovery owns the DWB exclusively — no concurrency, no
structure-modification dance, Chapter 10); beforelog_initialize /
log_recovery (redo gets non-torn homes); aftervacuum_initialize
(the data-volume descriptors the repair writes through are already
mounted). The empty-DWB + coherent-pages state (Invariant 9-A) is the
contract handed to redo.
dwb_Global.vdes must be NULL_VOLDES on entry — the global DWB is not
yet attached. This is asserted, not handled: a non-null vdes is a boot
ordering bug.
The outer if plus the inner power-of-2 test cover every restart shape
(taxonomy in §9.10 item 2). The DWB-volume-absent case (Branch A
false) — a clean shutdown deleted it, or this is first boot — falls
straight through to dwb_create: nothing to repair. Branch B-false
matters most: the source comment notes a valid DWB size must be a power
of 2, so a zero or non-power-of-2 count means the volume was created but
never coherently flushed (crash mid-creation) — recovery is skipped, no
home touched, control still proceeds to dismount/unformat/recreate.
Single recovery block regardless of original block count. Recovery
calls dwb_create_blocks (thread_p, 1, num_dwb_pages, &rcv_block): one
DWB_BLOCK sized to the entire on-disk DWB, not the configured
per-block size from Chapter 3 — recovery only needs every page laid out
contiguously so a single fileio_read_pages fills it.
The shared end: label (every error goto and the success path) frees
p_dwb_ordered_slots, finalizes and frees rcv_block, and returns
error_code. On error the old DWB volume is intentionally left on
disk (source comment: “Do not remove the old file if an error
occurs”) — a half-destroyed DWB would lose the only good copies, so the
operator can retry. fileio_unformat runs only on the Branch A success
path.
Once Branch B is taken, the whole DWB volume is read into the recovery
block’s write_buffer in one shot, and each slot’s identity is rebuilt
from io_page->prv — the volid/pageid/LSA triple a producer stamped when
it staged the page (Chapter 4):
rcv_block->count_wb_pages = num_dwb_pages; /* every page counts at recovery */
Because the home VPID lives inside the page image, reconstruction is
a plain field copy — there is no external slot index on disk.
count_wb_pages is set to the full page count so every slot counts.
9.4 dwb_block_create_ordered_slots — sort by (VPID, LSA)
*p_ordered_slots_length = block->count_wb_pages +1; /* includes sentinel */
dwb_compare_slots orders lexicographically by vpid.volid,
vpid.pageid, lsa.pageid, lsa.offset (LSA fields use 64-bit INT64
diffs to avoid overflow; VPID fields use int diffs), so two slots
with the same home VPID land adjacent, smaller LSA (older) first.
Invariant 9-B (ordered-slot adjacency). After
dwb_block_create_ordered_slots, duplicate home pages are contiguous
and ascending in LSA. The dedup loop and the debug checker depend on
this; they only ever compare neighbours i-1/i.
The sentinel is not part of the sorted region.qsort orders only
the first count_wb_pages elements; the sentinel sits at index
count_wb_pages, past the sorted range, and is never passed to the
comparator (a null-VPID slot would sort to the front if compared, but
it never is). It exists only so dwb_write_block’s loop over
ordered_slots_length = count_wb_pages + 1 terminates on a null slot;
the dedup and sanity loops bound at count_wb_pages and never touch it.
A page appears twice when a crash interrupts a flush: some slots hold the
current flush, others the previous. The dedup loop invalidates the
stale copy so it never overwrites the good one.
For an adjacent same-VPID pair: s1->lsa < s2->lsa nulls s1.
Equal LSA (dirtied without advancing LSA) falls back to
position_in_block — producers fill ascending positions, so the earlier
position held the most recent flush: keep the lower, null the higher.
Nulling a VPID is this chapter’s “ignore this slot”: every downstream
loop skips VPID_ISNULL.
9.6 dwb_debug_check_dwb — the duplicate detector (NDEBUG-gated)
Compiled only in debug builds (#if !defined (NDEBUG)), this is a
belt-and-suspenders check that the dedup left no genuine duplicate
behind. It walks neighbours i-1/i:
if (is_page_corrupted) continue; /* one corrupt -> not a real dup */
// ... same error_code-checked check on slot i; two memcmp vs a zeroed page skip uninit slots ...
assert (false); /* genuine duplicate -> bug */
}
}
Bothfileio_page_check_corruption calls capture error_code and
return error_code on failure (the second is condensed above) — a
corruption-check failure short-circuits the checker. Otherwise a same-VPID
neighbour is accepted only if at least one side is corrupted, or if either
side memcmps equal to a freshly zeroed iopage (an uninitialised slot);
a pair of sane, distinct, identical-VPID pages trips assert (false)
because §9.5 should already have nulled one. Release builds omit the
function — purely a development guard on Invariant 9-B.
9.7 dwb_check_data_page_is_sane — deciding what to repair
The heart of recovery: for each surviving slot, read the current home
page, decide via its page checksum whether the DWB copy must overwrite
it, and count recoverable pages so §9.8 can skip the write when nothing
is torn. “Torn” here means fails the page-checksum check
(fileio_page_check_corruption) — a page half-written when the crash hit
has an inconsistent checksum.
flowchart TB
S["slot i"] --> N{"VPID null /\nfd null /\npageid >= vol_pages?"}
N -- yes --> SKIP["skip"]
N -- no --> R["fileio_read home"]
R -- "read or check err" --> RERR["return error_code"]
R -- ok --> C{"home checksum ok?"}
C -- "yes, sane" --> CLEAN["null slot VPID +\nfileio_initialize_res"]
C -- "no, check err" --> RERR
C -- "no, torn" --> D{"DWB copy checksum ok?"}
D -- "no, corrupt" --> FATAL["assert_release false\nreturn ER_FAILED"]
D -- "yes, sane" --> REC["num_recoverable_pages++"]
Figure 9-1 — per-slot decision in dwb_check_data_page_is_sane. Every
arm ends in skip, clean-skip, an upward error_code return (the
fileio_read failure or eitherfileio_page_check_corruption
failing), fatal ER_FAILED, or recover.
num_recoverable_pages++; /* DWB sane, home torn -> repair */
}
*p_num_recoverable_pages = num_recoverable_pages;
Branch notes: the (VPID,LSA) sort groups slots by volume, so the
descriptor / page-count lookups run once per volume; the
pageid >= vol_pages skip covers a page that lived in the DWB but was
never allocated in the data volume (extended, then crashed before the
volume grew). On any of the three upward-error arms the caller’s end:
cleanup still runs.
Invariant 9-C (overwrite only the checksummed-torn). A home page is
overwritten from the DWB only when the home page fails its checksum
(fileio_page_check_corruption reports corrupt) and the DWB copy
passes its checksum. Sane homes are removed from the write set —
recovery repairs torn writes, it never “restores” a good page to an
older image. If both home and DWB copy fail the checksum, recovery
aborts with ER_FAILED rather than write a known-bad page.
9.8 Writing repairs, fsync, then destroy and recreate
for (i =0; i <rcv_block->count_flush_volumes_info; i++)
{
if (fileio_synchronize (thread_p, rcv_block->flush_volumes_info[i].vdes, NULL, true)
== NULL_VOLDES)
{ error_code = ER_FAILED; goto end; }
}
rcv_block->count_flush_volumes_info=0;
}
assert (rcv_block->count_flush_volumes_info ==0);
dwb_write_block (Chapter 6) writes each non-null ordered slot to its
home (volid, pageid). The recovery call passes false, false for the
trailing flags file_sync_helper_can_flush / remove_from_hash:
recovery fsyncs itself below and has no slots hash to evict from
(vdes == NULL_VOLDES), whereas the runtime flush passes true, true.
Every repaired data volume (in flush_volumes_info) is then
fileio_synchronized so repairs are durable before the DWB volume is
destroyed; the assert confirms the list is drained.
Branch A then runs its tail (the §9.2 skeleton): fileio_dismount +
fileio_unformat close and delete the DWB volume, and dwb_create
builds a fresh empty DWB to the configured parameter size, not the
recovered size — where an operator-changed double_write_* size takes
effect, and why §9.2 tolerates a differing recovered count. This realises
Invariant 9-A: torn homes overwritten and fsynced, old DWB gone, clean
empty DWB ready for the daemons. Only then does log_recovery run redo
against trusted homes.
9.9 Cross-check notes (raw analysis vs current source)
§9.1 excerpt is condensed, not verbatim. The real restart path has
oid_set_root between vacuum_initialize and the recovery call, and
the SERVER_MODE block also calls
parallel_query::worker_manager_global::get_manager ().init () after
dwb_daemons_init. The excerpt keeps the load-bearing statements; the
elided ones do not affect DWB ordering.
Latent format-string mismatch in the dedup dwb_log. The source’s
“Found duplicates” log line uses a format string with three%d
specs ("...positions = (%d,%d) %d\n") but passes only two
arguments (s1->position_in_block, s2->position_in_block). The third
spec reads an undefined vararg. It is harmless in practice (only fires
under dwb_check_logging()), but it is a real defect in the source,
not in this chapter’s summary of the logic. The chapter’s §9.5 excerpt
omits the dwb_log calls for brevity and so does not reproduce the
bug.
dwb_write_block is defined far above the restart-recovery
functions in the source file, not adjacent to them. The
position-hint table records its verified definition line.
Position in boot is the whole point. Recovery runs after
vacuum_initialize and before dwb_daemons_init / log_initialize,
so it owns the DWB alone and hands log_recovery coherent homes
(Invariant 9-A). See cubrid-recovery-manager-detail.md for the redo
phase that consumes this guarantee.
Two outer branches cover all cases. No DWB volume -> dwb_create
only; present but zero / not power-of-2 -> skip repair, recreate;
present and power-of-2 -> the real recovery path.
The volume is self-describing. One DWB_BLOCK of num_dwb_pages,
one fileio_read_pages, slot VPID/LSA rebuilt from io_page->prv —
no external on-disk index.
Sort then dedup.dwb_block_create_ordered_slots sorts only the
count_wb_pages real slots by (VPID, LSA) (Invariant 9-B); the
sentinel stays past the sorted region as a loop terminator. The dedup
loop nulls the older copy, breaking LSA ties on position_in_block.
dwb_debug_check_dwb re-asserts the result in debug builds.
Torn is a checksum verdict, and you overwrite only the torn
(Invariant 9-C).dwb_check_data_page_is_sane runs
fileio_page_check_corruption on the home page: sane home -> null the
slot (never overwrite); home torn + DWB copy checksum-valid -> count
it for repair; home torn + DWB copy also corrupt -> fatal ER_FAILED;
page beyond the data volume -> skip. A read or corruption-check
failure propagates error_code to the end: cleanup.
Repairs durable before the DWB is destroyed.dwb_write_block
(both trailing flags false) overwrites torn homes from their
checksum-valid DWB images, then fileio_synchronize fsyncs every
touched volume before fileio_unformat deletes the DWB. On any error
the old DWB volume is left on disk for a retry.
Recreated to the configured size, not the recovered size, via
dwb_create — so an operator can resize the DWB across a restart.
Chapter 10: Structure Modification Wait Queues and Teardown
Chapters 4 through 9 traced the main page lifecycle. This chapter covers the paths that branch off it — resizing, recreating, or destroying the DWB at runtime, plus the wait-queue primitives every waiting path shares. No new structs (Chapter 1 has DWB_WAIT_QUEUE / DWB_WAIT_QUEUE_ENTRY, Chapter 2 the position-with-flags word); this chapter dissects the operations: how CUBRID quiesces the buffer, drains every in-flight block, wakes every waiter, and rebuilds or releases the volume — without losing a page or deadlocking a producer.
Create, recreate, and destroy all bracket their work between dwb_starts_structure_modification and dwb_ends_structure_modification, turning a many-producer two-daemon system into a single-threaded critical section without a coarse lock. The gate is the MODIFY_STRUCTURE flag bit in position_with_flags (Chapter 2): producers that try to flush check it and route into dwb_wait_for_strucure_modification instead. Branch-complete flow:
Gate — the CAS retry loop, the only subtle branch:
} while (!ATOMIC_CAS_64 (&dwb_Global.position_with_flags, local_current_position_with_flags, new_position_with_flags));
Drain (SERVER_MODE only, no-op in SA_MODE) — thread_sleep (20) while blocks_flush_counter > 0 or dwb_flush_block_daemon_is_running () or dwb_file_sync_helper_daemon_is_running ().
Steal — once the daemons are idle the modifier is the only DWB accessor, so it claims the helper’s parked block (if any):
Scan — re-read position_with_flags, then find the oldest (min_version) write-started block → start_block_no.
Sweep — from start_block_no modulo DWB_NUM_TOTAL_BLOCKS: if write-started, dwb_flush_block (..., /*file_sync_helper=*/false, ...) to fsync inline; on error return error_code (flag stays set). Loop until blocks_count == 0.
Post — assert (DWB_GET_BLOCK_STATUS (...) == 0): no write-started blocks remain; ascending order matches recovery (Chapter 9).
Invariant — at most one structure modifier at a time (I2). The DWB_IS_MODIFYING_STRUCTURE test sits inside the CAS loop: of two racing threads one wins; the loser re-reads, sees the bit, returns ER_FAILED. Break it and two threads both enter dwb_destroy_internal and double-free dwb_Global.blocks. The asymmetry is intentional — a second modifier fails fast, a second producer waits.
Invariant — an error during the drain leaves the flag set (I9). A failed dwb_flush_block returns without clearing the flag (only dwb_ends_structure_modification clears it, which the failed caller never reaches) — deliberate fail-stop. Break it and the DWB reopens with a stale block; instead it stays frozen and the next restart replays the volume (Chapter 9).
ATOMIC_TAS_64 (&dwb_Global.position_with_flags, new_position_with_flags); /* plain TAS, no contender */
dwb_signal_structure_modificated (thread_p); /* wake global-queue waiters */
DWB_ENDS_MODIFYING_STRUCTURE clears the flag bit; the assert re-affirms the single-modifier invariant (with the flag set no other thread may write the word, so the live value still equals the modifier’s snapshot); ATOMIC_TAS_64 publishes it — a plain TAS, since there is no contender to lose to; and dwb_signal_structure_modificated wakes global-queue waiters (Section 10.5).
10.3 dwb_recreate, dwb_destroy — composing the bracket
Both entry points sequence the bracket around dwb_destroy_internal / dwb_create_internal. dwb_recreate is called by the system-parameter machinery when PRM_ID_DWB_SIZE or PRM_ID_DWB_BLOCKS change at runtime. Three branches: dwb_starts_structure_modification fails → return error_code; else if DWB_IS_CREATED, dwb_destroy_internal tears down the old size; then dwb_create_internal (thread_p, dwb_Volume_name, ...), goto end on error — end: runs dwb_ends_structure_modification. “Destroy then create under one flag” makes a resize atomic (old DWB or new, never a torn intermediate); it reuses the existing dwb_Volume_name, so only the in-memory geometry changes.
dwb_destroy has three branches, labelled inline below; branch 3 (dwb_daemons_destroy) is what dwb_recreate omits because the recreated buffer still needs the daemons:
dwb_daemons_destroy (); /* <- branch 3: full shutdown only, not on recreate */
#endif
dwb_destroy_internal (Chapter 3) does the heavy lifting: dwb_destroy_wait_queue (&dwb_Global.wait_queue, ...) drains the global queue, destroys the mutex, dwb_finalize_block-s every block, destroys the slots hashmap, dismounts/unformats the volume, then CASes the word back to reset+ended-creation. The SA-vs-server split is the #if defined(SERVER_MODE) guards — SA_MODE has no daemons, so dwb_daemons_destroy and the Section 10.1 drain loop are no-ops.
The wait queue is a hand-rolled singly-linked list with an embedded free list, reused for per-block waiters (DWB_BLOCK::wait_queue) and global structure-modification waiters (dwb_Global.wait_queue); all primitives are non-reentrant, the caller serializing with the mutex. dwb_init_wait_queue zeroes both lists. dwb_make_wait_queue_entry allocates with a free-list fast path (pop free_list if non-empty, else malloc — only the malloc branch fails, returning NULL). dwb_block_add_wait_queue_entry wraps it, appending at the tail with the empty-queue branch (head == NULL sets both head and tail).
dwb_block_disconnect_wait_queue_entry is the most branch-dense primitive — it unlinks by data pointer (or the head if data == NULL), with four exit branches labelled inline:
Invariant — head/tail/count agree (I6). After every primitive count equals the nodes reachable from head, tail is the last node (or NULL iff empty), tail->next == NULL. disconnect fixes tail on tail removal; add updates it on append. Break it and dwb_signal_waiting_threads’ drain loop (ending on head == NULL) loops forever or wakes a freed entry.
dwb_block_free_wait_queue_entry does not free memory — it returns immediately on a NULL entry (which is what makes dwb_remove_wait_queue_entry safe when disconnect returns NULL), then pushes the entry onto free_list after an optional callback; data is left intact for debugging. dwb_remove_wait_queue_entry composes disconnect + free under an optional mutex — the building block the timeout and interrupt paths call.
dwb_signal_waiting_threads drains the entire queue — while (wait_queue->head != NULL) dwb_remove_wait_queue_entry (wait_queue, NULL, NULL, dwb_signal_waiting_thread); — passing NULL for the inner mutex (already held) and NULL data so each call removes the head. dwb_signal_waiting_thread is the per-entry callback:
The two guards (inline) skip a cleared entry and prevent re-waking a thread that already timed out. dwb_set_status_resumed is the other callback (timeout path): it sets resume_status to THREAD_DWB_QUEUE_RESUMEDwithout waking — the thread woke itself. Both are no-ops outside SERVER_MODE. dwb_signal_block_completion and dwb_signal_structure_modificated are thin wrappers naming the occasion (per-block queue on flush completion, Chapter 6; global queue on modification end, Section 10.2), both delegating to dwb_signal_waiting_threads. dwb_destroy_wait_queue is the teardown: it signals all waiters, then walks free_listfree()-ing every recycled entry — the only primitive that frees to the heap.
10.6 The waiter side — dwb_wait_for_block_completion
A producer needing a write-started block flushed parks here. It re-checks under the mutex (the block may have completed between read and lock — early return NO_ERROR), enqueues itself (dwb_block_add_wait_queue_entry; NULL → alloc error), suspends with a 20 ms timeout, then handles three resume outcomes:
Five exit branches: already-done, alloc-error, timeout, shutdown interrupt, normal wake. The global-queue twin dwb_wait_for_strucure_modification (Section 10.1) mirrors this with two differences: a 10 ms timeout, and its timeout branch passes NULL as the callback (not dwb_set_status_resumed) — the global waiter does not bother re-stamping its own resume_status, since after a modification ends it simply re-checks the flag and proceeds.
Invariant — a waiter enqueues once, dequeues exactly once (I5). The block mutex serializes enqueue; the resume branch decides who dequeues (timeout/interrupt self-dequeue, normal relies on the signaller). Break it and a double-dequeue corrupts the list; prevented because dwb_block_disconnect_wait_queue_entry returns NULL for an already-gone entry and dwb_block_free_wait_queue_entry then frees nothing.
10.7 Observability and the temp-volume short-circuit
dwb_is_created is the cheap public predicate every writer consults — one atomic read plus the DWB_IS_CREATED macro. dwb_get_volume_name gates the name on it, returning NULL when the DWB is absent so callers don’t reference a stale path.
The temp-volume short-circuit lives in the caller: in file_io.c, dwb_is_created () routes a write through dwb_add_page only after fileio_traverse_permanent_volume confirms the target is permanent. Temporary volumes fall through to a direct write — discarded on restart, so the DWB write-amplification (see the high-level companion) would be waste. The DWB’s own volume is isolated by reserved id LOG_DBDWB_VOLID (= LOG_DBFIRST_VOLID - 22 in log_volids.hpp), in the negative system-volume range outside the data-volume id space the producer scans — letting the same predicate exclude the DWB volume from being buffered into itself.
10.8 Cross-cutting invariants — tying the chapters together
One flag bit serializes structural change (I2): a second modifier fails fast (ER_FAILED), producers wait — the asymmetry is intentional.
The modifier drains before touching memory (I7), flushes write-started blocks ascending and fsyncs inline; a failed drain is fail-stop, leaving the flag set so recovery replays the volume (I9).
dwb_recreate is destroy-then-create under one flag (atomic resize); dwb_destroy adds the never-created short-circuit and daemon teardown atop the starts-failed early return.
The wait queue recycles via free_list (free is a no-op on NULL); only dwb_destroy_wait_queue frees to the heap; head/tail/count stay consistent (I6).
Resume has three branches (I5) — timeout, shutdown interrupt, normal signal — so a waiter enqueues once and dequeues once; dwb_signal_waiting_thread null-guards the entry data.
The DWB excludes itself and temp volumes (I8): only permanent volumes are buffered, its own volume sits at reserved LOG_DBDWB_VOLID; dwb_is_created / dwb_get_volume_name are the observability gates.