What is Red Stuff?

Red Stuff is the magic behind the Walrus Protocol.

It’s a unique 2D encoding algorithm designed to power Walrus, a next-generation decentralized storage protocol. By addressing the limitations of current storage approaches, Red Stuff ensures fast, reliable, and scalable storage without the usual trade-offs.

To understand what makes Red Stuff a game-changer, let’s first look at how decentralized storage works today and where other methods fall short.

Approach 1 – full file replication

This approach, used by protocols like Arweave, stores the complete file on all nodes.

This ensures high availability since all nodes are incentivised to always have the complete file.

Problems with this approach

High storage overhead. Each node storing the file must store a complete replica, leading to an excessive use of storage space.

Inefficient resource usage. Sending complete files to all nodes consumes significant bandwidth, and recovering data can be slow and costly.

Approach 2 – encode and share

This method, used by protocols like Storj and Sia, breaks files into smaller pieces (slivers) using Reed-Solomon (RS) codes. 

Instead of storing complete copies like in the full file replication approach, slivers are distributed across different nodes, significantly reducing storage overhead. The file can be fully reconstructed as long as enough slivers are available.

Problems with this approach

High computational overhead. RS coding requires complex mathematical operations to split files into slivers and reconstruct them later, significantly increasing processing power and resource consumption.

Slow data retrieval. Encoding, decoding, and verifying slivers take longer, making uploads and downloads less efficient than full replication.

Approach 3 – Red Stuff

Red Stuff is the innovative 2D encoding algorithm that powers Walrus. It combines the best features of previous decentralized storage methods while solving their biggest challenges.

Built to be Byzantine Fault Tolerant, Red Stuff offers fast, reliable, and scalable storage without the inefficiencies of earlier systems.

How Red Stuff overcomes these problems

Red Stuff achieves low computational overhead

Where previous methods like RS coding rely on heavy mathematical operations for encoding and decoding, Red Stuff simplifies the process with lightweight XOR operations.

This leads to faster encoding and decoding speeds while lowering computational demands, making it quick and resource-efficient.

Red Stuff maintains low storage overhead

Unlike full replication, which duplicates entire files across all nodes, Red Stuff uses a 2D encoding process.

It splits data into primary and secondary slivers using simple XOR operations. This approach achieves a replication factor of 4-5x, making it far more efficient than other methods while maintaining strong data availability.

Red Stuff ensures efficient data retrieval

Red Stuff’s 2D encoding uses different thresholds for each dimension. 

The primary dimension allows asynchronous writes with minimal signatures, while the secondary dimension ensures data can be recovered efficiently, even if some slivers are lost.

This setup not only speeds up data retrieval but also makes Walrus the first to offer asynchronous storage proofs, ensuring resilience even in the presence of faulty nodes.

The new standard for storage

Red Stuff offers the best of both worlds.

By eliminating high storage overhead, reducing computational costs, and enabling faster data retrieval, it removes the trade-offs that have long plagued decentralized storage solutions.