State Storage Optimizations

Efficient state storage is critical for high-performance blockchains like Celestium, which aims to achieve high throughput and low-latency execution. Optimizing how blockchain state is stored and accessed directly impacts the speed of transaction processing, block validation, and node synchronization.

CelesDb is the foundation of Celestium’s state storage, but additional optimizations are applied to maximize performance and reduce resource consumption.

---

Challenges in Blockchain State Storage

Blockchain state represents account balances, smart contract data, and other state variables. This state grows continuously as transactions modify it. Key challenges include:

• Frequent Random Reads and Writes: Each transaction may read and write to different parts of the state.

• Large State Size: As the network grows, the state can reach terabytes in size.

• Efficient Proof Generation: Merkle proofs are needed for light clients, requiring efficient state root computation.

• Concurrency: Supporting parallel execution demands concurrent state reads and writes without data corruption.

---

Key State Storage Optimizations in Celestium

Optimization	Description
Trie Node Caching	Frequently accessed parts of the Merkle Patricia Trie are cached in memory to reduce disk reads.
Compact Trie Node Encoding	Trie nodes are stored in a compressed format to minimize disk usage and reduce I/O overhead.
Node Preloading	Anticipates state access patterns during execution and preloads relevant trie nodes into memory.
Batch State Updates	State changes from multiple transactions are batched into a single write operation, reducing I/O cost.
Parallel State Reads	Allows multiple transactions to read different parts of the state concurrently during parallel execution.
Asynchronous State Access	State reads and writes are non-blocking, leveraging async I/O (e.g., io_uring on Linux).
Sparse Storage Layout	Sparse Merkle Trie layout reduces storage overhead for accounts and contracts with minimal state.
Write-ahead Buffering	State changes are first written to a buffer before being committed to disk, minimizing expensive random writes.

---

Trie Node Caching

To minimize disk I/O, Celestium employs an adaptive caching strategy:

• Hot State Cache: Frequently accessed accounts and contract storage slots are kept in memory.

• Trie Node Cache: Patricia Trie nodes involved in recent reads or writes are cached to speed up future accesses.

• Execution Context Cache: Intermediate state during parallel and optimistic execution is kept in a local context cache before merging.

This reduces redundant disk reads and accelerates transaction execution.

---

Compact Trie Node Encoding

Each node in the Merkle Patricia Trie is represented in a compact format to:

• Reduce storage size on disk.

• Accelerate deserialization during state reads.

• Minimize the number of I/O operations.

For example, nodes with short paths are packed into a single structure, reducing the need to traverse multiple disk blocks.

---

Parallel State Reads

Parallel transaction execution requires the state database to serve concurrent read operations efficiently:

• Lock-free reads: Reads are allowed in parallel without locking.

• Snapshot isolation: Transactions read a snapshot of the state at the block start, avoiding conflicts.

• Trie Segmentation: Trie nodes are segmented so that reads on different branches can proceed independently.

This ensures that parallel execution threads can access state concurrently without blocking each other.

---

Batch State Updates

Instead of writing state changes to disk for each transaction, Celestium:

1. Accumulates state updates during block execution.

2. Batches writes into a single I/O operation at the end of execution.

3. Merges updates efficiently into the Patricia Trie.

This drastically reduces the number of random disk writes and improves throughput.

---

Asynchronous State Access

Building on CelesDb’s async I/O capabilities, state reads and writes are designed to be non-blocking:

• Concurrent state access requests are issued during transaction execution.

• Other operations continue while waiting for disk reads/writes.

• Async completion allows results to be processed as soon as they are ready.

This is crucial for parallel execution, ensuring CPU resources are never idle waiting for I/O.

---

Node Preloading

Celestium leverages execution pre-analysis to predict which parts of the state will be accessed:

• Static analysis of smart contract code predicts state dependencies.

• State prefetching loads trie nodes into memory before execution begins.

• Reduces random disk reads during execution.

This optimization works particularly well when transactions interact with the same contracts frequently.

---

Sparse Storage Layout

Accounts and contract storage are designed to handle sparse keys efficiently. This is particularly useful for:

• Contracts with large mappings but few actual entries.

• Accounts with sparse nonces or balances across multiple tokens.

Sparse layouts ensure that empty state entries do not consume disk space unnecessarily.

---

Write-ahead Buffering

During block execution, state changes are buffered in-memory before being written to disk:

• Reduces disk writes during speculative execution (parallel and optimistic execution).

• Accelerates transaction throughput, especially for blocks with frequent state updates.

• Ensures crash consistency by logging buffered updates before committing.

---

Benefits of State Storage Optimizations

Benefit	Impact
Faster Transaction Execution	Reduced disk I/O and faster state access enable low-latency execution.
Improved Parallelism	Concurrent reads and async writes support Celestium’s parallel and optimistic execution models.
Reduced Disk Wear	Batched writes and caching reduce the number of random disk writes, extending SSD lifespan.
Lower Memory Footprint	Compact encoding and adaptive caching optimize memory usage.
Scalability	Efficient handling of large state ensures Celestium can scale to millions of accounts and contracts.

---

Comparison to Traditional Ethereum Clients

Aspect	Celestium State Storage	Traditional Ethereum Clients
Trie Node Caching	Adaptive hot-state and trie caching	Basic caching, limited optimization
Parallel State Reads	Fully supported	Mostly sequential
Async I/O	Full async (io_uring support)	Blocking I/O
Batch Writes	Optimized write batching	Per-transaction writes
Sparse Storage	Efficient handling of sparse state	No special optimization

---

Final Thoughts

State storage optimizations in Celestium are purpose-built to complement parallel and asynchronous execution models. These enhancements ensure that state access is never a bottleneck, allowing Celestium to achieve high throughput without sacrificing Ethereum compatibility.

By combining custom database design (CelesDb) with advanced storage techniques, Celestium can scale to support large-scale decentralized applications while maintaining low latency and high efficiency.