Bab - Verifiable partial data for peer-to-peer systems.

Tutorial

In this tutorial, we're going to use the bab_rs crate to:

Create a digest from an in-memory slice
Hash data incrementally
Persist Bab-annotated data to the filesystem

Prerequisites

A basic knowledge of the Rust programming language and executing commands in the terminal will be helpful for completing this tutorial. Some of the steps below also require cargo to be installed.

Setup

Create a new directory on your filesystem and name it something like bab_tutorial.
Using your terminal, run cargo init within the newly created directory.
After than, run cargo add bab_rs ufotofu pollster.

Create a digest from an in-memory slice

Firstly we will use batch_hash and William3Digest to create digests.

Open src/main.rs, delete the contents, and insert the following:

use bab_rs::Hasher;
use bab_rs::HasherWrite;
use bab_rs::William3Digest;
use bab_rs::William3Hasher;
use bab_rs::batch_hash;

fn main() {
    // Have a byte slice in memory? Use batch_hash.
    let mut digest = William3Digest::default();
    let slice = b"Hello Bab!";
    batch_hash(slice, &mut digest);
    println!("The digest: {digest:?}");

    // Want to hash things incrementally? Use William3Hasher.
    let mut hasher = William3Hasher::new();
    hasher.write(b"He");
    hasher.write(b"ll");
    hasher.write(b"o ");
    hasher.write(b"Ba");
    hasher.write(b"b!");
    let incremental_digest = hasher.finish();

    assert_eq!(digest, incremental_digest);
    println!("That's the basics!");   
}

In your terminal, run cargo run, and you should see the following output:

The digest: William3Digest(BabDigest([204, 87, 211, 1, 126, 183, 80, 116, 132, 221, 190, 164, 18, 153, 171, 205, 124, 102, 249, 238, 30, 184, 170, 245, 91, 10, 131, 209, 173, 144, 95, 218]))
That's the basics!

Store some Bab-annotated data

In this step we will use SingleSliceStore to ingest some data, annotate it with Bab data, and persist it to the filesystem.

Insert the following text into src/main.rs:

use bab_rs::Hasher;
use bab_rs::HasherWrite;
use bab_rs::William3Digest;
use bab_rs::William3Hasher;
use bab_rs::batch_hash;

use bab_rs::generic::storage::backend_filesystem::FileBackend;
use bab_rs::generic::storage::backend_filesystem::KeyState;
use bab_rs::storage::SingleSliceStore;
use ufotofu::IntoConsumer;
use ufotofu::IntoProducer;

fn main() {
    // Have a byte slice in memory? Use batch_hash.
    let mut digest = William3Digest::default();
    let slice = b"Hello Bab!";
    batch_hash(slice, &mut digest);
    println!("The digest: {digest:?}");

    // Want to hash things incrementally? Use William3Hasher.
    let mut hasher = William3Hasher::new();
    hasher.write(b"He");
    hasher.write(b"ll");
    hasher.write(b"o ");
    hasher.write(b"Ba");
    hasher.write(b"b!");
    let incremental_digest = hasher.finish();

    assert_eq!(digest, incremental_digest);
    println!("That's the basics!");

    // Persistent storage.
    pollster::block_on(async {
        // Store some data and create a digest for it.
        let mut slice_producer = [b'B', b'a', b'b', b'!'].into_producer();
        let mut key_state = KeyState::new();
        let (mut store, store_digest) = SingleSliceStore::<FileBackend>::create_and_initialise(
            &mut key_state,
            "./tutorial".into(),
            4,
            &mut slice_producer,
        )
        .await
        .unwrap();

        // And compare that digest to a in-memory slice...
        let mut batch_digest = William3Digest::default();
        let slice = b"Bab!";
        batch_hash(slice, &mut batch_digest);

        assert_eq!(batch_digest, store_digest.into());
        println!("We persisted some data with a digest!");

        // Load the data from the store
        let mut consumer = Vec::with_capacity(2).into_consumer();
        store.get_data(&mut consumer, 0, 2).await.unwrap();

        assert_eq!(consumer.as_slice(), b"Ba");
        println!("We got some data out of the store!");
    });
}

In your terminal, run cargo run, and you should see the following output:

The digest: William3Digest(BabDigest([204, 87, 211, 1, 126, 183, 80, 116, 132, 221, 190, 164, 18, 153, 171, 205, 124, 102, 249, 238, 30, 184, 170, 245, 91, 10, 131, 209, 173, 144, 95, 218]))
That's the basics!
We persisted some data with a digest!
We got some data out of the store!

Reload and delete the store

Finally we will use SingleSliceStore::load to load a Bab store from the filesystem, and then SingleSliceStore::load to delete it.

Insert the following text into src/main.rs:

use bab_rs::Hasher;
use bab_rs::HasherWrite;
use bab_rs::William3Digest;
use bab_rs::William3Hasher;
use bab_rs::batch_hash;

use bab_rs::generic::storage::backend_filesystem::FileBackend;
use bab_rs::generic::storage::backend_filesystem::KeyState;
use bab_rs::storage::SingleSliceStore;
use ufotofu::IntoConsumer;
use ufotofu::IntoProducer;

fn main() {
    // Have a byte slice in memory? Use batch_hash.
    let mut digest = William3Digest::default();
    let slice = b"Hello Bab!";
    batch_hash(slice, &mut digest);
    println!("The digest: {digest:?}");

    // Want to hash things incrementally? Use William3Hasher.
    let mut hasher = William3Hasher::new();
    hasher.write(b"He");
    hasher.write(b"ll");
    hasher.write(b"o ");
    hasher.write(b"Ba");
    hasher.write(b"b!");
    let incremental_digest = hasher.finish();

    assert_eq!(digest, incremental_digest);
    println!("That's the basics!");

    // Persistent storage.
    pollster::block_on(async {
        // Store some data and create a digest for it.
        let mut slice_producer = [b'B', b'a', b'b', b'!'].into_producer();
        let mut key_state = KeyState::new();
        let (mut store, store_digest) = SingleSliceStore::<FileBackend>::create_and_initialise(
            &mut key_state,
            "./tutorial".into(),
            4,
            &mut slice_producer,
        )
        .await
        .unwrap();

        // And compare that digest to a in-memory slice...
        let mut batch_digest = William3Digest::default();
        let slice = b"Bab!";
        batch_hash(slice, &mut batch_digest);

        assert_eq!(batch_digest, store_digest.into());
        println!("We persisted some data with a digest!");

        // Load the data from the store
        let mut consumer = Vec::with_capacity(2).into_consumer();
        store.get_data(&mut consumer, 0, 2).await.unwrap();

        assert_eq!(consumer.as_slice(), b"Ba");
        println!("We got some data out of the store!");
        
        // Drop the store and reload it.
        store.flush().await.unwrap();
        core::mem::drop(store);
        let mut store = SingleSliceStore::<FileBackend>::load(&mut key_state, &"./tutorial".into())
            .await
            .unwrap()
            .unwrap();
        
        let mut another_consumer = Vec::with_capacity(2).into_consumer();
        store.get_data(&mut another_consumer, 2, 2).await.unwrap();
        
        assert_eq!(another_consumer.as_slice(), b"b!");
        println!("We got some data out of the store after loading it!");
        
        // Delete the store's contents.
        SingleSliceStore::<FileBackend>::delete(&mut key_state, &"./tutorial".into())
            .await
            .unwrap();
        
        let mut yet_another_consumer = Vec::<u8>::with_capacity(4).into_consumer();
        assert!(
            store
                .get_data(&mut yet_another_consumer, 0, 4)
                .await
                .is_err()
        );
        
        core::mem::drop(store);
        
        assert!(
            SingleSliceStore::<FileBackend>::load(&mut key_state, &"./tutorial".into())
                .await
                .unwrap()
                .is_none()
        );
        println!("We really deleted some data!");
    });
}

In your terminal, run cargo run, and you should see the following output:

The digest: William3Digest(BabDigest([204, 87, 211, 1, 126, 183, 80, 116, 132, 221, 190, 164, 18, 153, 171, 205, 124, 102, 249, 238, 30, 184, 170, 245, 91, 10, 131, 209, 173, 144, 95, 218]))
That's the basics!
We persisted some data with a digest!
We got some data out of the store!
We got some data out of the store after loading it!
We really deleted some data!

Summary

In this tutorial, we used bab_rs to create digests and store data:

We used batch_hash and William3Hasher to create digests from in-memory slices,
We used SingleSliceStore to annotate some data with Bab metadata and persist it all to the filesystem,
We used SingleSliceStore::load to load Bab-annotated data from the filesystem,
And finally, we used SingleSliceStore::delete to delete that data.

To learn more about bab_rs's APIs (including how to incrementally store data to the filesystem), please see the Rust APIs.