Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,12 +183,6 @@ Snapshot Count:
Snapshot Restore:
kvs_tool -o snapshotrestore -s 1

Get KVS Filename:
kvs_tool -o getkvsfilename -s 1

Get Hash Filename:
kvs_tool -o gethashfilename -s 1

---------------------------------------

Create Test Data:
Expand Down
124 changes: 124 additions & 0 deletions docs/class_diagram.puml
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
@startuml Persistency class diagram

package "kvs_builder" {
~class KvsInner {
~parameters: KvsParameters
~data: KvsData
}

+class KvsBuilder {
{static} -KVS_POOL: list<KvsInner>
-parameters: KvsParameters

+new(instance_id: InstanceId): KvsBuilder
{static} +max_instances(): usize
+defaults(mode: KvsDefaults): KvsBuilder
+kvs_load(mode: KvsLoad): KvsBuilder
+backend_parameters(parameters: KvsMap): KvsBuilder
+build(): Kvs
}

KvsInner <-- KvsBuilder
}

package "kvs_api" {
+interface KvsApi {
+reset()
+reset_key(key: string)
+get_all_keys(): list<string>
+key_exists(key: string): bool
+get_value(key: string): KvsValue
+get_value_as<T>(key: string): T
+get_default_value(key: string): KvsValue
+is_value_default(key: string): bool
+set_value<T>(key: string, value: T)
+remove_key(key: string)
+flush()
+snapshot_count(): usize
+snapshot_max_count(): usize
+snapshot_restore(snapshot_id: SnapshotId)
}
}

package "kvs" {
+class Kvs {
-data: KvsData
-parameters: KvsParameters
-backend: KvsBackend

~new(data: KvsData, parameters: KvsParameters, backend: KvsBackend): Kvs
+parameters(): KvsParameters

+reset()
+reset_key(key: string)
+get_all_keys(): list<string>
+key_exists(key: string): bool
+get_value(key: string): KvsValue
+get_value_as<T>(key: string): T
+get_default_value(key: string): KvsValue
+is_value_default(key: string): bool
+set_value<T>(key: string, value: T)
+remove_key(key: string)
+flush()
+snapshot_count(): usize
+snapshot_max_count(): usize
+snapshot_restore(snapshot_id: SnapshotId)
}
}

KvsApi <|.. Kvs
KvsBuilder -- Kvs

package "kvs_backend_registry" {
+class KvsBackendRegistry {
{static} -REGISTERED_BACKENDS: map<string, KvsBackendFactory>

{static} ~from_name(name: string) -> Result<&Box<dyn KvsBackendFactory>, ErrorCode>
{static} ~from_parameters(parameters: KvsMap) -> Result<&Box<dyn KvsBackendFactory>, ErrorCode>

{static} +register(backend_factory: KvsBackendFactory)
}
}

KvsBackendRegistry .. KvsBuilder

package "kvs_backend" {
+interface KvsBackend {
+load_kvs(instance_id: InstanceId, snapshot_id: SnapshotId): KvsMap
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the proposed API, the user always have to wait until whole KVS is populated only then values are accessible.
Why not to allow direct access to the value and satisfy requirements for earliest possible availability?
With restriction that only read access is possible eg. load_value(key: string): KvsValue

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, currently it is assumed that storage is fully memory-mapped. You can add it as a topic for requirements discussion.

+load_defaults(instance_id: InstanceId): KvsMap
+flush(instance_id: InstanceId, kvs_map: KvsMap)
+snapshot_count(instance_id: InstanceId): usize
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I didn't get the whole idea behind the snapshots, my understanding and some questions:

  • User can create snapshot in order to "conserve" the current values
  • After values in working copy were modified, user can restore the "conserved" values with snapshot_restore(instance_id: InstanceId, snapshot_id: SnapshotId)
  • Is the snapshot_restore() intended to overwrite the working copy? I see that it return KvsMap so probably not.
  • Do we need the snapshot_restore() at all? load_kvs() can take snapshot ID as well
  • Who is taking care of book-keeping the SnapshotIds? If this is the backend, then we need something like get_snapshots() which will return list of existing snapshot IDs
  • snapshot_delete() will be needed as well
  • How are the snapshots created? Do we need something like snapshot_create()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • snapshot_restore is intended to overwrite current state, You still need to flush.
  • No strong opinion on whether we need snapshot_restore at all. It's just load with additional error checks, we can also provide it as an already implemented trait method, since I cannot imagine other behavior than current.
  • SnapshotId behavior should be documented. Right now it is assumed that "0" means current, and "1", "2", "123" means 1 ago, 2 ago, 123 ago.
  • snapshot_delete is already required, but not implemented. This function will break the SnapshotId behavior explained above, we'll need to reconsider.
  • We don't need snapshot_create, since flush stores current state as "0". Previously existing snapshots are rotated ("0"->"1", "1"->"2") or removed (if snapshot_max_count=3, then "2" is removed).

Those are valid points, but should be taken into consideration once requirements are modified.

+snapshot_max_count(): usize
+snapshot_restore(instance_id: InstanceId, snapshot_id: SnapshotId): KvsMap
}

+interface KvsBackendFactory {
+create(parameters: KvsMap): KvsBackend
}
}

package "json_backend" {
~class JsonBackend {
+load_kvs(instance_id: InstanceId, snapshot_id: SnapshotId): KvsMap
+load_defaults(instance_id: InstanceId): KvsMap
+flush(instance_id: InstanceId, kvs_map: KvsMap)
+snapshot_count(instance_id: InstanceId): usize
+snapshot_max_count(): usize
+snapshot_restore(instance_id: InstanceId, snapshot_id: SnapshotId): KvsMap
}

~class JsonBackendFactory {
+create(parameters: KvsMap): KvsBackend
}
}

KvsBackend <-- Kvs

KvsBackendRegistry --> KvsBackendFactory
KvsBackendFactory -- KvsBackend

KvsBackend <|.. JsonBackend
KvsBackendFactory <|.. JsonBackendFactory
JsonBackendFactory -- JsonBackend

@enduml
111 changes: 111 additions & 0 deletions src/rust/rust_kvs/examples/backend_registration.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
//! Example for custom backend registration.
//! - Implementation of `KvsBackend` traits.
//! - Registration of custom backend.
//! - Creation of KVS instance utilizing custom backend.

use rust_kvs::prelude::*;
use tempfile::tempdir;

/// Mock backend implementation.
/// Only `load_kvs` is implemented.
struct MockBackend;

impl KvsBackend for MockBackend {
fn load_kvs(
&self,
_instance_id: InstanceId,
_snapshot_id: SnapshotId,
) -> Result<KvsMap, ErrorCode> {
println!("`load_kvs` used");
Ok(KvsMap::new())
}

fn load_defaults(&self, _instance_id: InstanceId) -> Result<KvsMap, ErrorCode> {
unimplemented!()
}

fn flush(&self, _instance_id: InstanceId, _kvs_map: &KvsMap) -> Result<(), ErrorCode> {
unimplemented!()
}

fn snapshot_count(&self, _instance_id: InstanceId) -> usize {
unimplemented!()
}

fn snapshot_max_count(&self) -> usize {
unimplemented!()
}

fn snapshot_restore(
&self,
_instance_id: InstanceId,
_snapshot_id: SnapshotId,
) -> Result<KvsMap, ErrorCode> {
unimplemented!()
}
}

/// Mock backend factory implementation.
struct MockBackendFactory;

impl KvsBackendFactory for MockBackendFactory {
fn create(&self, _parameters: &KvsMap) -> Result<Box<dyn KvsBackend>, ErrorCode> {
Ok(Box::new(MockBackend))
}
}

fn main() -> Result<(), ErrorCode> {
// Temporary directory.
let dir = tempdir()?;
let dir_string = dir.path().to_string_lossy().to_string();

// Register `MockBackendFactory`.
KvsBackendRegistry::register("mock", || Box::new(MockBackendFactory))?;

// Build KVS instance with mock backend.
{
let instance_id = InstanceId(0);
let parameters = KvsMap::from([("name".to_string(), KvsValue::String("mock".to_string()))]);
let builder = KvsBuilder::new(instance_id)
.backend_parameters(parameters)
.defaults(KvsDefaults::Ignored);
let kvs = builder.build()?;

println!(
"KVS instance with mock backend - parameters: {:?}",
kvs.parameters()
);
}

// Build KVS instance with JSON backend - default parameters.
{
let instance_id = InstanceId(1);
let builder = KvsBuilder::new(instance_id).defaults(KvsDefaults::Ignored);
let kvs = builder.build()?;

println!(
"KVS instance with default JSON backend - parameters: {:?}",
kvs.parameters()
);
}

// Build KVS instance with JSON backend - `working_dir` set.
{
let instance_id = InstanceId(2);
let parameters = KvsMap::from([
("name".to_string(), KvsValue::String("json".to_string())),
("working_dir".to_string(), KvsValue::String(dir_string)),
]);
let builder = KvsBuilder::new(instance_id)
.backend_parameters(parameters)
.defaults(KvsDefaults::Ignored);
let kvs = builder.build()?;

println!(
"KVS instance with JSON backend - parameters: {:?}",
kvs.parameters()
);
}

Ok(())
}
8 changes: 6 additions & 2 deletions src/rust/rust_kvs/examples/basic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ fn main() -> Result<(), ErrorCode> {
// Temporary directory.
let dir = tempdir()?;
let dir_string = dir.path().to_string_lossy().to_string();
let backend_parameters = KvsMap::from([
("name".to_string(), KvsValue::String("json".to_string())),
("working_dir".to_string(), KvsValue::String(dir_string)),
]);

// Instance ID for KVS object instances.
let instance_id = InstanceId(0);
Expand All @@ -20,7 +24,7 @@ fn main() -> Result<(), ErrorCode> {
// `kvs_load` is explicitly set to `KvsLoad::Optional`, but this is the default value.
// KVS files are not required.
let builder = KvsBuilder::new(instance_id)
.dir(dir_string.clone())
.backend_parameters(backend_parameters.clone())
.kvs_load(KvsLoad::Optional);
let kvs = builder.build()?;

Expand Down Expand Up @@ -65,7 +69,7 @@ fn main() -> Result<(), ErrorCode> {
// Build KVS instance for given instance ID and temporary directory.
// `kvs_load` is set to `KvsLoad::Required` - KVS files must already exist from previous KVS instance.
let builder = KvsBuilder::new(instance_id)
.dir(dir_string)
.backend_parameters(backend_parameters)
.kvs_load(KvsLoad::Required);
let kvs = builder.build()?;

Expand Down
6 changes: 5 additions & 1 deletion src/rust/rust_kvs/examples/custom_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,10 @@ fn main() -> Result<(), ErrorCode> {
// Temporary directory.
let dir = tempdir()?;
let dir_string = dir.path().to_string_lossy().to_string();
let backend_parameters = KvsMap::from([
("name".to_string(), KvsValue::String("json".to_string())),
("working_dir".to_string(), KvsValue::String(dir_string)),
]);

// Create initial example object.
let object = Example {
Expand Down Expand Up @@ -218,7 +222,7 @@ fn main() -> Result<(), ErrorCode> {
let kvs = KvsBuilder::new(InstanceId(0))
.kvs_load(KvsLoad::Ignored)
.defaults(KvsDefaults::Ignored)
.dir(dir_string)
.backend_parameters(backend_parameters)
.build()?;

// Serialize and set object.
Expand Down
6 changes: 5 additions & 1 deletion src/rust/rust_kvs/examples/defaults.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ fn main() -> Result<(), ErrorCode> {
// Temporary directory.
let dir = tempdir()?;
let dir_string = dir.path().to_string_lossy().to_string();
let backend_parameters = KvsMap::from([
("name".to_string(), KvsValue::String("json".to_string())),
("working_dir".to_string(), KvsValue::String(dir_string)),
]);

// Instance ID for KVS object instances.
let instance_id = InstanceId(0);
Expand All @@ -44,7 +48,7 @@ fn main() -> Result<(), ErrorCode> {
// Build KVS instance for given instance ID and temporary directory.
// `defaults` is set to `KvsDefaults::Required` - defaults are required.
let builder = KvsBuilder::new(instance_id)
.dir(dir_string)
.backend_parameters(backend_parameters)
.defaults(KvsDefaults::Required);
let kvs = builder.build()?;

Expand Down
10 changes: 7 additions & 3 deletions src/rust/rust_kvs/examples/snapshots.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@ use rust_kvs::prelude::*;
use tempfile::tempdir;

fn main() -> Result<(), ErrorCode> {
// Temporary directory.
// Temporary directory and common backend.
let dir = tempdir()?;
let dir_string = dir.path().to_string_lossy().to_string();
let backend_parameters = KvsMap::from([
("name".to_string(), KvsValue::String("json".to_string())),
("working_dir".to_string(), KvsValue::String(dir_string)),
]);

// Instance ID for KVS object instances.
let instance_id = InstanceId(0);
Expand All @@ -17,7 +21,7 @@ fn main() -> Result<(), ErrorCode> {
println!("-> `snapshot_count` and `snapshot_max_count` usage");

// Build KVS instance for given instance ID and temporary directory.
let builder = KvsBuilder::new(instance_id).dir(dir_string.clone());
let builder = KvsBuilder::new(instance_id).backend_parameters(backend_parameters.clone());
let kvs = builder.build()?;

let max_count = kvs.snapshot_max_count() as u32;
Expand All @@ -38,7 +42,7 @@ fn main() -> Result<(), ErrorCode> {
println!("-> `snapshot_restore` usage");

// Build KVS instance for given instance ID and temporary directory.
let builder = KvsBuilder::new(instance_id).dir(dir_string.clone());
let builder = KvsBuilder::new(instance_id).backend_parameters(backend_parameters);
let kvs = builder.build()?;

let max_count = kvs.snapshot_max_count() as u32;
Expand Down
9 changes: 9 additions & 0 deletions src/rust/rust_kvs/src/error_code.rs
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,15 @@ pub enum ErrorCode {

/// Instance parameters mismatch
InstanceParametersMismatch,

/// Requested unknown backend
UnknownBackend,

/// Backend already registered
BackendAlreadyRegistered,

/// Invalid backend parameters.
InvalidBackendParameters,
}

impl From<std::io::Error> for ErrorCode {
Expand Down
Loading