slate/duplicate_bag
DETS duplicate bag tables: multiple values per key, including duplicates.
Duplicate bag tables are like bag tables but allow storing identical key-value pairs multiple times.
Example
import gleam/dynamic/decode
import slate/duplicate_bag
let assert Ok(table) = duplicate_bag.open("events.dets",
key_decoder: decode.string, value_decoder: decode.string)
let assert Ok(Nil) = duplicate_bag.insert(table, "click", "button_a")
let assert Ok(Nil) = duplicate_bag.insert(table, "click", "button_a")
let assert Ok(["button_a", "button_a"]) =
duplicate_bag.lookup(table, "click")
let assert Ok(Nil) = duplicate_bag.close(table)
Types
An open DETS duplicate bag table with typed keys and values.
pub opaque type DuplicateBag(k, v)
Values
pub fn close(
table: DuplicateBag(k, v),
) -> Result(Nil, slate.DetsError)
Close the table, flushing all pending writes to disk.
The table handle must not be used after closing.
pub fn delete_all(
from table: DuplicateBag(k, v),
) -> Result(Nil, slate.DetsError)
Delete all entries and keep the table open.
pub fn delete_key(
from table: DuplicateBag(k, v),
key key: k,
) -> Result(Nil, slate.DetsError)
Delete all values for the given key.
Returns Ok(Nil) even if the key does not exist.
pub fn delete_object(
from table: DuplicateBag(k, v),
key key: k,
value value: v,
) -> Result(Nil, slate.DetsError)
Delete all occurrences of a specific key-value pair from the table.
In a duplicate bag, this removes every copy of the exact pair. Other values (or different duplicates) for the same key are preserved.
import gleam/dynamic/decode
let assert Ok(table) = duplicate_bag.open("events.dets",
key_decoder: decode.string, value_decoder: decode.string)
let assert Ok(Nil) = duplicate_bag.insert(table, "click", "btn_a")
let assert Ok(Nil) = duplicate_bag.insert(table, "click", "btn_a")
let assert Ok(Nil) = duplicate_bag.insert(table, "click", "btn_b")
let assert Ok(Nil) = duplicate_bag.delete_object(table, "click", "btn_a")
// Only "btn_b" remains
pub fn fold(
over table: DuplicateBag(k, v),
from initial: acc,
with callback: fn(acc, k, v) -> acc,
) -> Result(acc, slate.DetsError)
Fold over all entries. Order is unspecified.
Returns Error(DecodeErrors(_)) if any entry does not match the
expected types. The fold stops at the first decode error. If the callback
raises, the exception is re-raised.
pub fn fold_results(
over table: DuplicateBag(k, v),
from initial: acc,
with callback: fn(
acc,
Result(#(k, v), List(decode.DecodeError)),
) -> acc,
) -> Result(acc, slate.DetsError)
Call your callback once for each entry, in an unspecified order.
Pass the decoded entry as Ok(#(key, value)), or its decode errors as
Error(decode_errors). Each duplicate copy counts as an entry.
The callback returns the next accumulator value.
Unlike fold, this function continues after a decode error.
The function returns Ok(accumulator) when it finishes. A table error,
such as TableDoesNotExist, makes the function return Error(error).
If the callback raises an exception, that exception propagates to the caller.
Examples
Skip entries that fail to decode:
duplicate_bag.fold_results(table, [], fn(acc, entry) {
case entry {
Ok(#(key, value)) -> [#(key, value), ..acc]
Error(_) -> acc
}
})
Collect decoded entries and decode errors separately:
duplicate_bag.fold_results(table, #([], []), fn(acc, entry) {
case entry {
Ok(#(key, value)) -> #([#(key, value), ..acc.0], acc.1)
Error(errors) -> #(acc.0, [errors, ..acc.1])
}
})
pub fn info(
table: DuplicateBag(k, v),
) -> Result(slate.TableInfo, slate.DetsError)
Get the file size in bytes, entry count, and absolute path of an open table.
Returns Error(TableDoesNotExist) if the table is no longer open.
pub fn insert(
into table: DuplicateBag(k, v),
key key: k,
value value: v,
) -> Result(Nil, slate.DetsError)
Insert a key-value pair. Duplicates are stored separately.
Unlike set and bag tables, duplicate bag does not provide an insert_new
function because duplicate key-value pairs are explicitly allowed.
pub fn insert_list(
into table: DuplicateBag(k, v),
entries entries: List(#(k, v)),
) -> Result(Nil, slate.DetsError)
Insert multiple key-value pairs.
pub fn lookup(
from table: DuplicateBag(k, v),
key key: k,
) -> Result(List(v), slate.DetsError)
Look up all values for a key, including duplicates, in an unspecified order.
Returns Ok([]) if the key does not exist. This differs from
set.lookup, which returns Error(NotFound) for missing keys.
Returns Error(DecodeErrors(_)) if any stored value does not match the
expected type.
pub fn member(
of table: DuplicateBag(k, v),
key key: k,
) -> Result(Bool, slate.DetsError)
Check if a key exists without returning the values.
pub fn open(
path: String,
key_decoder key_decoder: decode.Decoder(k),
value_decoder value_decoder: decode.Decoder(v),
) -> Result(DuplicateBag(k, v), slate.DetsError)
Open or create a DETS duplicate bag table at the given file path.
The parent directory must exist. This function uses AutoRepair and
ReadWrite.
Reads use your decoders to check stored data. Opening a table does not check all its entries.
import gleam/dynamic/decode
let assert Ok(table) = duplicate_bag.open("data/events.dets",
key_decoder: decode.string, value_decoder: decode.string)
pub fn open_with(
path path: String,
repair repair: slate.RepairPolicy,
key_decoder key_decoder: decode.Decoder(k),
value_decoder value_decoder: decode.Decoder(v),
) -> Result(DuplicateBag(k, v), slate.DetsError)
Open or create a DETS duplicate bag table with a specific repair policy.
The repair policy controls what happens when the table file was not closed cleanly, for example after an abnormal node shutdown:
AutoRepair: Attempt repair if needed. This is the default foropen.ForceRepair: Repair even if the file was closed properly.NoRepair: ReturnNeedsRepairif the file requires repair.
Make a backup before you repair a file with data you need to keep. Repair does not guarantee recovery of all data.
import gleam/dynamic/decode
import slate.{ForceRepair}
let assert Ok(table) = duplicate_bag.open_with(path: "data/events.dets",
repair: ForceRepair,
key_decoder: decode.string, value_decoder: decode.string)
pub fn open_with_access(
path path: String,
repair repair: slate.RepairPolicy,
access access: slate.AccessMode,
key_decoder key_decoder: decode.Decoder(k),
value_decoder value_decoder: decode.Decoder(v),
) -> Result(DuplicateBag(k, v), slate.DetsError)
Open a DETS duplicate bag table with repair and access mode options.
Use ReadOnly to open a table for reading only. Write operations
on a read-only table will return Error(AccessDenied(context)).
import gleam/dynamic/decode
import slate.{AutoRepair, ReadOnly}
let assert Ok(table) = duplicate_bag.open_with_access(path: "data/events.dets",
repair: AutoRepair, access: ReadOnly,
key_decoder: decode.string, value_decoder: decode.string)
let assert Ok(values) = duplicate_bag.lookup(table, key: "key")
// duplicate_bag.insert(table, "key", "val") would return Error(AccessDenied(context))
pub fn size(
of table: DuplicateBag(k, v),
) -> Result(Int, slate.DetsError)
Return the number of stored entries, not the number of distinct keys.
Each copy of a key-value pair counts as an entry.
pub fn sync(
table: DuplicateBag(k, v),
) -> Result(Nil, slate.DetsError)
Flush pending writes to disk without closing the table.
By default, DETS saves the table after three minutes without table access.
Call sync and handle its result when you need to write pending updates
to disk before continuing.
pub fn to_list(
from table: DuplicateBag(k, v),
) -> Result(List(#(k, v)), slate.DetsError)
Return all entries as a list in an unspecified order, including duplicates.
This loads the entire table into memory.
Returns Error(DecodeErrors(_)) if any entry does not match the
expected types.
pub fn with_exclusive_table(
path path: String,
repair repair: slate.RepairPolicy,
access access: slate.AccessMode,
wait wait: exclusive.Wait,
key_decoder key_decoder: decode.Decoder(k),
value_decoder value_decoder: decode.Decoder(v),
callback callback: fn(DuplicateBag(k, v)) -> Result(
a,
slate.DetsError,
),
) -> Result(a, exclusive.SessionError)
Use a table under exclusive ownership across OS processes.
All clients, including readers, must participate. Ownership covers open,
repair, the callback, and clean close. wait limits acquisition only.
Keep the handle within the callback; do not reopen or manually close it.
Callback errors do not roll back writes. Exceptions are re-raised after
cleanup; cleanup failures are logged. Unconfirmed cleanup can retain
ownership until VM exit.
Requires the native component and a qualified local filesystem. Symlinks are resolved; hard-linked data and unsafe sidecars are rejected. Data paths and persistent sidecars must remain stable while clients can run sessions.
pub fn with_table(
path path: String,
repair repair: slate.RepairPolicy,
access access: slate.AccessMode,
key_decoder key_decoder: decode.Decoder(k),
value_decoder value_decoder: decode.Decoder(v),
callback callback: fn(DuplicateBag(k, v)) -> Result(
a,
slate.DetsError,
),
) -> Result(a, slate.DetsError)
Use a table within a callback with repair and access mode options.
Select the repair policy and access mode as for open_with_access.
The callback must return a Result. ReadOnly requires an existing
file, and writes return Error(AccessDenied(context)). NoRepair returns
Error(NeedsRepair(context)) if the file was not closed cleanly.
If opening fails, returns the open error without calling the callback. Otherwise, runs the callback and attempts to close the table. If close fails after a successful callback, returns the close error. If both fail, returns the callback error. If the callback raises, attempts close before re-raising the original exception.
Keep the table handle within the callback; do not use it after cleanup. This cannot guarantee cleanup if the process is killed.
import gleam/dynamic/decode
import slate.{NoRepair, ReadOnly}
import slate/duplicate_bag
use table <- duplicate_bag.with_table(path: "data/events.dets",
repair: NoRepair, access: ReadOnly,
key_decoder: decode.string, value_decoder: decode.string)
duplicate_bag.lookup(table, key: "click")