Skip to content

Commit 7828482

Browse files
codeboostcodeboost
committed
Update README.md
1 parent 198e9bf commit 7828482

File tree

1 file changed

+208
-23
lines changed

1 file changed

+208
-23
lines changed

‎README.md

Lines changed: 208 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,224 @@
1-
# XITDB Clojure
1+
## Overview
22

3-
`xitdb-clj` is a Clojure native interface on top of the immutable database [xitdb-java](https://github.com/radarroark/xitdb-java).
3+
`xitdb-clj` is a database for efficiently storing and retrieving immutable, persistent data structures.
44

5+
It is a Clojure interface for [xitdb-java](https://github.com/radarroark/xitdb-java),
6+
itself a port of [xitdb](https://github.com/radarroark/xitdb), written in Zig.
57

6-
It allows you to work with the database as if it were a Clojure atom.
8+
`xitdb-clj` provides atom-like semantics when working with the database from Clojure.
79

8-
### Quick example
10+
### Experimental
911

10-
One code sample is worth a thousand words in a README:
12+
Code is still in early stages of 'alpha', things might change or break in future versions.
13+
14+
## Main characteristics
15+
16+
- Embeddable, tiny library.
17+
- Supports writing to a file as well as purely in-memory use.
18+
- Each transaction (done via `swap!`) efficiently creates a new "copy" of the database, and past copies can still be read from.
19+
- Reading/Writing to the database is extremely efficient, only the necessary nodes are read or written.
20+
- Thread safe. Multiple readers, one writer.
21+
- Append-only. The data you are writing is invisible to any reader until the very last step, when the top-level history header is updated.
22+
- No dependencies besides `clojure.core` and [xitdb-java](https://github.com/radarroark/xitdb-java).
23+
- All heavy lifting done by the bare-to-the-jvm java library.
24+
- Database files accessible from many other languages - all JVM based or languages which can natively interface with Zig (C, C++, Python, Rust, Go, etc)
25+
26+
## Architecture
27+
28+
XitDB-Clj builds on `xitdb-java` which implements:
29+
30+
- **Hash Array Mapped Trie (HAMT)** - For efficient map and set operations
31+
- **RRB Trees** - For vector operations with good concatenation performance
32+
- **Structural Sharing** - Minimizes memory usage across versions
33+
- **Copy-on-Write** - Ensures immutability while maintaining performance
34+
35+
The Clojure wrapper adds:
36+
- Idiomatic Clojure interfaces (`IAtom`, `IDeref`)
37+
- Automatic type conversion between Clojure and Java types
38+
- Thread-local read connections for scalability
39+
- Integration with Clojure's sequence abstractions
40+
41+
## Why you always wanted this in your life
42+
43+
### You already know how to use it!
44+
45+
For the programmer, a `xitdb` database behaves exactly like a Clojure atom.
46+
`reset!` or `swap!` to reset or update, `deref` or `@` to read.
1147

1248
```clojure
13-
(def db (xdb/xit-db "testing.db"))
14-
15-
(reset! db {:users {"1234" {:name "One Two Three Four" :address {:city "Barcelona"}}}})
16-
17-
;; Read the contents of the DB is if it were an atom
18-
@db
19-
; => {:users {"1234" {:name "One Two Three Four", :address {:city "Barcelona"}}}}
20-
21-
(swap! db update-in [:users "1234" :address] merge {:street "Gran Via" :postal-code "08010"})
22-
(get-in @db [:users "1234" :address :street])
49+
(def db (xdb/xit-db "my-app.db"))
50+
;; Use it like an atom
51+
(reset! db {:users {"alice" {:name "Alice" :age 30}
52+
"bob" {:name "Bob" :age 25}}})
53+
;; Read the entire database
54+
(common/materialize @db)
55+
;; => {:users {"alice" {:name "Alice", :age 30}, "bob" {:name "Bob", :age 25}}}
2356

24-
; => "Gran Via"
57+
(get-in @db [:users "alice" :age])
58+
;; => 30
59+
(swap! db assoc-in [:users "alice" :age] 31)
60+
61+
(get-in @db [:users "alice" :age])
62+
;; => 31
63+
```
64+
65+
## Data structures are read lazily from the database
66+
67+
Reading from the database returns wrappers around cursors in the database file:
68+
69+
```clojure
70+
(type @db) ;; => xitdb.hash_map.XITDBHashMap
2571
```
2672

73+
The returned value is a `XITDBHashMap` which is a wrapper around the xitdb-java's `ReadHashMap`,
74+
which itself has a cursor to the tree node in the database file.
75+
These wrappers implement the protocols for Clojure collections - vectors, lists, maps and sets,
76+
so they behave exactly like the Clojure native data structures.
77+
Any read operation on these types is going to return new `XITDB` types:
78+
79+
```clojure
80+
(type (get-in @db [:users "alice"])) ;; => xitdb.hash_map.XITDBHashMap
81+
```
82+
83+
So it will not read the entire nested structure into memory, but return a 'cursor' type, which you can operate upon
84+
using Clojure functions.
85+
86+
Use `materialize` to convert a nested `XITDB` data structure to a native Clojure data structure:
87+
88+
```clojure
89+
(materialize (get-in @db [:users "alice"])) ;; => {:name "Alice" :age 31}
90+
```
91+
92+
## No query language
93+
94+
Use `filter`, `group-by`, `reduce`, etc.
95+
If you want a query engine, `datascript` works out of the box, you can store the datoms as a vector in the db.
96+
97+
Here's a taste of how your queries could look like:
98+
```clojure
99+
(defn titles-of-songs-for-artist
100+
[db artist]
101+
(->> (get-in db [:songs-indices :artist artist])
102+
(map #(get-in db [:songs % :title]))))
103+
104+
(defn what-is-the-most-viewed-song? [db tag]
105+
(let [views (->> (get-in db [:songs-indices :tag tag])
106+
(map (:songs db))
107+
(map (juxt :id :views))
108+
(sort-by #(parse-long (second %))))]
109+
(get-in db [:songs (first (last views))])))
110+
111+
```
112+
113+
## History
114+
Since the database is immutable, all previous values are accessing by reading
115+
from the respective `history index`.
116+
The root data structure of a xitdb database is a ArrayList, called 'history'.
117+
Each transaction adds a new entry into this array, which points to the latest value
118+
of the database (usually a map).
119+
It is also possible to create a transaction which returns the previous and current
120+
values of the database, by setting the `*return-history?*` binding to `true`.
121+
122+
```clojure
123+
;; Work with history tracking
124+
(binding [xdb/*return-history?* true]
125+
(let [[history-index old-value new-value] (swap! db assoc :new-key "value")]
126+
(println "old value:" old-value)
127+
(println "new value:" new-value)))
128+
```
129+
130+
### Supported Data Types
131+
- **Maps** - Hash maps with efficient key-value access
132+
- **Vectors** - Array lists with indexed access
133+
- **Sets** - Hash sets with unique element storage
134+
- **Lists** - Linked lists and RRB tree-based linked array lists
135+
- **Primitives** - Numbers, strings, keywords, booleans, dates.
136+
137+
### Persistence Models
138+
- **File-based** - Data persisted to disk with crash recovery
139+
- **In-memory** - Fast temporary storage for testing or caching
140+
141+
142+
## Installation
143+
144+
Add to your `deps.edn`:
145+
146+
```clojure
147+
{:deps {org.clojure/clojure {:mvn/version "1.12.0"}
148+
io.github.radarroark/xitdb {:mvn/version "0.20.0"}
149+
;; Add your local xitdb-clj dependency here
150+
}}
151+
```
152+
153+
## Examples
154+
155+
### User Management System
156+
157+
```clojure
158+
(def user-db (xdb/xit-db "users.db"))
159+
160+
(reset! user-db {:users {}
161+
:sessions {}
162+
:settings {:max-sessions 100}})
163+
164+
;; Add a new user
165+
(swap! user-db assoc-in [:users "user123"]
166+
{:id "user123"
167+
:email "alice@example.com"
168+
:created-at (java.time.Instant/now)
169+
:preferences {:theme "dark" :notifications true}})
170+
171+
;; Create a session
172+
(swap! user-db assoc-in [:sessions "session456"]
173+
{:user-id "user123"
174+
:created-at (java.time.Instant/now)
175+
:expires-at (java.time.Instant/ofEpochSecond (+ (System/currentTimeMillis) 3600))})
176+
177+
;; Update user preferences
178+
(swap! user-db update-in [:users "user123" :preferences]
179+
merge {:language "en" :timezone "UTC"})
180+
```
181+
182+
### Configuration Store
183+
184+
```clojure
185+
(def config-db (xdb/xit-db "app-config.db"))
186+
187+
(reset! config-db
188+
{:database {:host "localhost" :port 5432 :name "myapp"}
189+
:cache {:ttl 3600 :max-size 1000}
190+
:features #{:user-registration :email-notifications :analytics}
191+
:rate-limits [{:path "/api/*" :requests-per-minute 100}
192+
{:path "/upload" :requests-per-minute 10}]})
193+
194+
;; Enable a new feature
195+
(swap! config-db update :features conj :real-time-updates)
196+
197+
;; Update database configuration
198+
(swap! config-db assoc-in [:database :host] "db.production.com")
199+
```
200+
201+
## Performance Characteristics
202+
203+
- **Read Operations**: O(log₃₂ n) for maps and vectors due to trie structure
204+
- **Write Operations**: O(log₃₂ n) with structural sharing for efficiency
205+
- **Memory Usage**: Minimal overhead with automatic deduplication of identical subtrees
206+
- **Concurrency**: Thread-safe with optimized read-write locks
207+
208+
## Testing
209+
210+
Run the test suite:
211+
212+
```bash
213+
clojure -M:test
214+
```
27215

28-
Yeah, it's extremely cool. Once you start using it, you might pinch yourself when you realise that most of the database-related data munging simply disappears from your code.
29-
There's no translation between your app domain to database and back, it's just Clojure all the way to the disk platter or NAND flash memory chips.
30216

31-
### Clojure data structures, persisted
217+
## Contributing
32218

33-
`xitdb-java` provides an efficient implementation of several data structures:
34-
`HashMap` and `ArrayList` are based on the hash array mapped trie from Phil Bagwell. `LinkedArrayList` is based on the RRB tree, also from Phil Bagwell.
35-
If it rings a bell it's because Clojure's data structures are also built on them.
219+
This project welcomes contributions. Please ensure all tests pass and follow the existing code style.
36220

221+
## License
37222

38-
### Current status
223+
[License information needed]
39224

0 commit comments

Comments
 (0)