Agenda

1. Why a storage engine — what the layer must guarantee
2. The layered stack — one diagram of the whole thing
3. From the bottom up — disk_manager → page_buffer + DWB → heap → btree → ehash → overflow
4. Crossing concerns — TDE, the WAL contract, recovery's view
5. Where to read next — symbol-level pointers

Closing: Beyond CUBRID — InnoDB, PostgreSQL, RocksDB, Oracle.

Why a storage engine, what it must guarantee

The storage engine is everything below the catalog and the locator and above the operating system's filesystem. It takes byte streams from the OS and produces fixed-size pages, records, indexes — under four hard guarantees:

• Durability. A committed write must survive a crash. Implemented by Write-Ahead Logging (WAL) — log records are forced to disk before the dirty page they describe.
• Atomicity at page grain. A single page write is either fully old or fully new on disk — never a half-old / half-new "torn" page. CUBRID buys this with the Double Write Buffer.
• Fast point reads + range scans. Heap pages give random access by OID; B+Tree leaves give ordered scans by key.
• Concurrent access without corruption. Multiple workers may read or mutate the same page; the page buffer hands out a custom read / write / flush latch on each in-memory frame.

Everything in the rest of the talk is one of these four guarantees, paid for in code.

The layered stack

• Strict layering. Each layer knows only the one below it.
• Page buffer is the choke point. Every record / key layer talks to it.

Every record layer talks to the page buffer, not the disk

The single structural rule that makes WAL ordering enforceable.

• The page buffer is the only module allowed to call pread/pwrite against a CUBRID data volume (via disk_manager + file_io). The heap, the B+Tree, the catalog, ehash — none of them ever touch a file descriptor directly.
• Mutation order. A heap insert / B+Tree split / ehash bucket split (1) acquires the page's PGBUF_LATCH in WRITE, (2) mutates the in-memory frame, (3) produces a log record routed through the prior list to the log manager, then (4) marks the page dirty and releases the latch.
• Flush order. The page buffer's flush path refuses to write a dirty page until the log manager confirms the page's last-modifying LSA is durable. ARIES's log first, then page rule, made structural.
• Page latch ≠ transactional lock. The latch is short, embedded in the BCB, microsecond-scale. Transactional locks live in the lock manager (see the Lock Manager deck).

If a record layer ever bypassed the page buffer, WAL ordering would collapse. The architecture forbids it by construction.

disk_manager — volumes, sectors, files, pages

The OS-file boundary. Four levels:

• Permanent vs temporary purpose split. Temp files (sort spills, hash partitions) live on a separate volume class and bypass WAL — they never need to survive a crash.
• Two-step sector reservation. A growing file (1) reserves N sectors from the in-memory disk cache, then (2) commits the reservation into the on-disk sector allocation table (STAB) under an ARIES nested top-action. Concurrent writers cooperate without serializing on the STAB.
• Adaptive volume extension. When the cache runs short, disk_volume_expand / disk_add_volume grow the current volume or add a new one — same nested top-action discipline.

Vocabulary: VPID, sector, file, volume. Every other doc in the section reuses it.

double_write_buffer — torn-write protection

A torn page is half-old / half-new on disk after a crash mid-write. WAL can recover a coherent before-state, not a Frankenstein. The DWB defeats this.

• Volume. A separate permanent file (<db>_dwb), 512 KiB – 32 MiB, split into 1–32 power-of-two blocks. LOG_DBDWB_VOLID is reserved for it.
• Producer side. A position-with-flags 64-bit atomic word coordinates concurrent producers. A filled block goes to the flush daemon.
• Recovery side. At restart, dwb_load_and_recover_pages reads each staged copy, checks its LSA, and if the home page is torn — fails its checksum — replaces the home page from the DWB copy. All before log replay starts.

overflow_file — big-record + big-OID-list spill

One symbol-level module (src/storage/overflow_file.c) serves three different spillover needs.

• One overflow file per containing object. Pages allocated from it are owned by exactly that heap / B+Tree — no cross-tenant overflow.
• Single-linked page chain per logical record. Each page header carries a "next-page" VPID; the first page also carries the record length.
• Reference at the home slot. Heap uses REC_BIGONE (one VPID); B+Tree uses the leaf_rec.ovfl pointer or an embedded VPID for overflow keys.
• Vacuum-aware. When MVCC vacuum reclaims a big record, the overflow chain is freed page-by-page under WAL.

TDE — encrypt-at-rest at the page boundary

Transparent Data Encryption: per-page encrypt-on-flush / decrypt-on-read. Largely orthogonal to the rest of the engine.

• Two-level key hierarchy. A Master Key (MK), supplied by the DBA at startup, lives outside the database in a <db>_keys file. The MK wraps three per-database Data Encryption Keys (DEK) — DATA, LOG, TEMP. Rotating the MK rewraps the DEKs; the data pages are untouched.
• Cipher & mode. AES-256-CTR or ARIA-256-CTR. CTR mode lets a page be encrypted in one parallel pass, with no length growth.
• Per-page nonce. LSA for permanent pages, an atomic counter for temp, logical pageid for log. The CTR invariant — (key, nonce) never repeats — falls out for free.
• Encryption boundary. Page header (FILEIO_PAGE_RESERVED with pflag, tde_nonce) stays plaintext — the buffer manager must read pflag to know whether to decrypt. The tail watermark also stays plaintext (torn-write check works on ciphertext too). Body in between is encrypted.
• Per-file granularity. TDE is a tablespace-style dial — each FILE_HEADER carries its own TDE_ALGORITHM; file_alloc stamps the algorithm onto each new page's pflag.

The WAL contract — page_buffer log_manager

The single rule that makes ARIES recovery work.

• Log first, then page. The buffer manager refuses to flush a dirty page until the log manager confirms oldest_unflush_lsa is durable.
• Every layer participates. Disk manager logs sector / volume changes (RVDK_*); heap logs record mutations (RVHF_*); B+Tree logs node mutations (RVBT_*); ehash logs directory + bucket changes (RVEH_*); overflow file logs page-chain extensions.
• The dependency is structural. Without it, replaying redo against a page newer than the log would be impossible — the storage engine would be unrecoverable.

Recovery's view of the storage stack

Three passes (ARIES). The storage stack participates in all three.

• DWB handshake runs before analysis. Any home page that fails its checksum is replaced from its DWB copy. Redo then sees a coherent before-state.
• MVCC vacuum is a heap-page mutation in its own right. It takes the WRITE latch and emits RVHF_* records, so vacuum survives crash recovery the same way as user DML.