You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Oct 1, 2021. It is now read-only.
Copy file name to clipboardExpand all lines: README.md
+65-68Lines changed: 65 additions & 68 deletions
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@
10
10
11
11
> Migration framework for versioning of JS IPFS Repo
12
12
13
-
This package takes inspiration from similar tool used by Go-IPFS: [fs-repo-migrations](https://github.com/ipfs/fs-repo-migrations/)
13
+
This package is inspired by the [go-ipfs repo migration tool](https://github.com/ipfs/fs-repo-migrations/)
14
14
15
15
## Lead Maintainer
16
16
@@ -34,16 +34,16 @@ This package takes inspiration from similar tool used by Go-IPFS: [fs-repo-migra
34
34
35
35
## Background
36
36
37
-
As JS-IPFS evolves and new technologies, algorithms and data structures are being incorporated it is necessary to
38
-
enable users easy transition between versions. Different versions of JS-IPFS may expect different structure or content
39
-
of the IPFS repo (see: [IPFS repo spec](https://github.com/ipfs/specs/tree/master/repo), [JS implementation](https://github.com/ipfs/js-ipfs-repo) ).
40
-
For that reason IPFS repo is versioned and this package provides framework to create migrations which transits
41
-
one version of IPFS repo into next/previous one.
42
37
43
-
This framework provides:
38
+
As js-ipfs evolves and new technologies, algorithms and data structures are incorporated it is necessary to
39
+
enable users to transition between versions. Different versions of js-ipfs may expect a different IPFS repo structure or content (see: [IPFS repo spec](https://github.com/ipfs/specs/tree/master/repo), [JS implementation](https://github.com/ipfs/js-ipfs-repo) ).
40
+
So the IPFS repo is versioned, and this package provides a framework to create migrations to transition
41
+
from one version of IPFS repo to the next/previous version.
42
+
43
+
This framework:
44
44
* Handles locking/unlocking of repository
45
45
* Defines migrations API
46
-
* Executes and reports migrations in both direction: forward and backward
46
+
* Executes and reports migrations in both directions: forward and backward
On how to migrate your repository using CLI, you can find the recommended steps in[how to run migrations](./run.md) tutorial.
86
+
To migrate your repository using the CLI, see the [how to run migrations](./run.md) tutorial.
87
87
88
-
**For tools that build on top of `js-ipfs` and run mainly in the browser environment, be aware of disabling automatic
89
-
migrations as user does not have any other way how to run the migrations because of lack of CLI in browser. In such
90
-
a case, you should provide a way how to trigger migrations manually.**
88
+
**For tools that build on top of `js-ipfs` and run mainly in the browser environment, be aware that disabling automatic
89
+
migrations leaves the user with no way to run the migrations because there is no CLI in the browser. In such
90
+
a case, you should provide a way to trigger migrations manually.**
91
91
92
92
### Writing migration
93
93
@@ -100,112 +100,109 @@ Migrations are one of those things that can be extremely painful on users. At th
100
100
101
101
#### Architecture of migrations
102
102
103
-
All migrations are placed in `/migrations` folder. Each folder there represents one migration that follows migration
103
+
All migrations are placed in the `/migrations` folder. Each folder there represents one migration that follows the migration
104
104
API.
105
105
106
-
All migrations are collected in `/migrations/index.js`, which should not be edited manually is it is regenerated on
107
-
every run of `jsipfs-migrations add` (or the manual changes should follow same style of modifications).
108
-
**The order of migrations is important and the migrations have to be sorted in the growing order**.
106
+
All migrations are collected in `/migrations/index.js`, which should not be edited manually. It is regenerated on
107
+
every run of `jsipfs-migrations add` (manual changes should follow the same style of modifications).
108
+
**The order of migrations is important and migrations must be sorted in ascending order**.
109
109
110
-
Each migration has to follow this API. It has to export object in its `index.js` that has following properties:
110
+
Each migration must follow this API. It must export an object in its `index.js` that has following properties:
111
111
112
-
*`version` (int) - Number that represents the version into which will be the repo migrated to (eq. `migration-8` will move the repo into version 8).
112
+
*`version` (int) - Number that represents the version which the repo will migrate to (eg. `migration-8` will move the repo to version 8).
113
113
*`description` (string) - Brief description of what the migrations does.
114
-
*`migrate` (function) - Function that on execution will perform the migration, see signature of this function bellow.
115
-
*`revert` (function) - If defined then this function will be used to revert the migration to previous version. Otherwise it is assumed that it is not possible to revert this migration.
114
+
*`migrate` (function) - Function that performs the migration (see signature of this function below)
115
+
*`revert` (function) - If defined then this function will revert the migration to the previous version. Otherwise it is assumed that it is not possible to revert this migration.
116
116
117
117
##### `migrate(repoPath, isBrowser)`
118
118
119
119
_Do not confuse this function with the `require('ipfs-repo-migrations').migrate()` function that drives the whole migration process!_
120
120
121
121
Arguments:
122
122
*`repoPath` (string) - absolute path to the root of the repo
123
-
*`options` (object, optional) - object containing `IPFSRepo` options, that should be used to construct datastore instance.
124
-
*`isBrowser` (bool) - indicates if the migration is run in browser environment in oppose to NodeJS
123
+
*`options` (object, optional) - object containing `IPFSRepo` options, that should be used to construct a datastore instance.
124
+
*`isBrowser` (bool) - indicates if the migration is run in a browser environment (as opposed to NodeJS)
125
125
126
126
##### `revert(repoPath, isBrowser)`
127
127
128
128
_Do not confuse this function with the `require('ipfs-repo-migrations').revert()` function that drives the whole backward migration process!_
129
129
130
130
Arguments:
131
131
*`repoPath` (string) - path to the root of the repo
132
-
*`options` (object, optional) - object containing `IPFSRepo` options, that should be used to construct datastore instance.
133
-
*`isBrowser` (bool) - indicates if the migration is run in browser environment in oppose to NodeJS
132
+
*`options` (object, optional) - object containing `IPFSRepo` options, that should be used to construct the datastore instance.
133
+
*`isBrowser` (bool) - indicates if the migration is run in a browser environment (as opposed to NodeJS)
134
134
135
135
#### Browser vs. NodeJS environments
136
136
137
-
Migration might need to distinguish in what environment it runs (browser vs. NodeJS), for this reason there is the argument
137
+
The migration might need to distinguish in which environment it runs (browser vs. NodeJS). For this reason there is an argument
138
138
`isBrowser` passed to migrations functions. But with simple migrations it should not be necessary to distinguish between
139
-
these environments as datastore implementation will handle the main differences.
139
+
these environments as the datastore implementation will handle the main differences.
140
140
141
141
There are currently two main datastore implementations:
142
-
1.[`datastore-fs`](https://github.com/ipfs/js-datastore-fs) that is backed by file system and is used mainly in NodeJS environment
143
-
2.[`datastore-level`](https://github.com/ipfs/js-datastore-level) that is backed by LevelDB and is used mainly in browser environment
144
-
145
-
Both implementations share same API and hence are interchangeable.
142
+
1.[`datastore-fs`](https://github.com/ipfs/js-datastore-fs) that is backed by file system and is used mainly in the NodeJS environment
143
+
2.[`datastore-level`](https://github.com/ipfs/js-datastore-level) that is backed by LevelDB and is used mainly in the browser environment
146
144
147
-
When the migration is run in browser environment the `datastore-fs` is automatically replaced with `datastore-level` even
148
-
when it is directly imported (`require('datastore-fs')` will return `datastore-level` in browser). Because of this mechanism
149
-
with simple migrations you should not worry about difference between `datastore-fs` and `datastore-level` and by default
150
-
use the `datastore-fs` package (as the replace mechanism does not work vice vera).
145
+
Both implementations share the same API and hence are interchangeable.
146
+
147
+
When the migration is run in a browser environment, `datastore-fs` is automatically replaced with `datastore-level` even
148
+
when it is directly imported (`require('datastore-fs')` will return `datastore-level` in a browser).
149
+
So with simple migrations you shouldn't worry about the difference between `datastore-fs` and `datastore-level`
150
+
and by default use the `datastore-fs` package (as the replace mechanism does not work vice versa).
151
151
152
152
#### Guidelines
153
153
154
-
The recommended way to write a new migration is to first bootstrap an dummy migration using the CLI:
154
+
The recommended way to write a new migration is to first bootstrap a dummy migration using the CLI:
155
155
156
156
```sh
157
157
> jsipfs-migrations add
158
158
```
159
159
160
-
Afterwards new folder is created with bootstrapped migration. You can then simply fill in the required fields and
161
-
write the rest of migration!
162
-
163
-
The `node_modules` of the migration should be committed to the repo to ensure that the dependencies are resolved even in
164
-
far future, when the package might be removed from registry.
160
+
A new folder is created with the bootstrapped migration. You can then simply fill in the required fields and
161
+
write the rest of the migration!
165
162
166
163
#### Migration's dependencies
167
164
168
-
Size of `js-ipfs`'s bundle is crucial for distribution in browser environment, hence dependency management of all related
165
+
The size of the `js-ipfs` bundle is crucial for distribution in a browser environment, so dependency management of all related
169
166
packages is important.
170
167
171
-
If migration need to depend on some package, this dependency should be declared in the root's `package.json`. Author
172
-
of migrations should be thoughtful about adding dependencies that would significantly increase the size of the final bundle.
168
+
If a migration needs to depend on some package, this dependency should be declared in the root's `package.json`. The author
169
+
of the migration should be thoughtful about adding dependencies that would significantly increase the size of the final bundle.
173
170
174
171
Most of the migration's dependencies will most likely overlap with `js-ipfs`'s dependencies and hence should not introduce
175
-
any significant overhead, but it requires to keep the versions of these dependencies in sync with `js-ipfs`. For this
176
-
reason migrations should be well tested to ensure correct behaviour over its dependencies updates. It might happen
177
-
that update of some dependency could introduce breaking change, in such a case the next steps should be discussed with broader
172
+
any significant overhead, but it is necessary to keep the versions of these dependencies in sync with `js-ipfs`. For this
173
+
reason migrations should be well tested to ensure correct behaviour over dependency updates.
174
+
An update of some dependency could introduce breaking change. In such a case the next steps should be discussed with a broader
178
175
audience.
179
176
180
177
#### Integration with js-ipfs
181
178
182
-
When new migration is created, the repo version in [`js-ipfs-repo`](https://github.com/ipfs/js-ipfs-repo) should be updated with the new version,
183
-
together with updated version of this package. And afterwards propagate the updated version to `js-ipfs`.
179
+
When a new migration is created, the repo version in [`js-ipfs-repo`](https://github.com/ipfs/js-ipfs-repo) should be updated with the new version,
180
+
together with updated version of this package. Then the updated version should be propagated to `js-ipfs`.
184
181
185
182
#### Tests
186
183
187
-
If migration affects working of any of the following functionality, it has to provide tests for the following functions
184
+
If a migration affects any of the following functionality, it must provide tests for the following functions
188
185
to work under the version of the repo that it migrates to:
189
186
190
187
*`/src/repo/version.js`:`getVersion()` - retrieving repository's version
191
188
*`/src/repo/lock.js`:`lock()` - locking repository that uses file system
192
189
*`/src/repo/lock-memory.js`:`lock()` - locking repository that uses memory
193
190
194
-
Every mgration has to have a test coverage. Tests for migration should be placed in `/test/migrations/` folder. Most probably
195
-
you will have to plug the tests into `browser.js`/`node.js` if they require specific bootstrap on each platform.
191
+
Every migration must have test coverage. Tests for migrations should be placed in the`/test/migrations/` folder. Most probably
192
+
you will have to plug the tests into `browser.js`/`node.js` if they require specific bootstrapping on each platform.
196
193
197
194
#### Empty migrations
198
195
199
-
For inter-operable reasons with Go-IPFS it might be necessary just to bump a version of repo without any actual
200
-
modification as there might not be any changes needed in JS implementation. For that you can create "empty migration".
196
+
For interop with go-ipfs it might be necessary just to bump a version of a repo without any actual
197
+
modification as there might not be any changes needed in the JS implementation. For that purpose you can create an "empty migration".
201
198
202
-
The easiest way is to use the CLI for that:
199
+
The easiest way to do so is with the CLI:
203
200
204
201
```sh
205
202
> jsipfs-migrations add --empty
206
203
```
207
204
208
-
This will create empty migration with next version in line.
205
+
This will create an empty migration with the next version.
209
206
210
207
### Migrations matrix
211
208
@@ -218,48 +215,48 @@ This will create empty migration with next version in line.
Executes forward migration to specific version or if not specified to the latest version.
218
+
Executes a forward migration to a specific version, or to the latest version if a specific version is not specified.
222
219
223
220
**Arguments:**
224
221
225
222
*`path` (string, mandatory) - path to the repo to be migrated
226
223
*`options` (object, optional) - options for the migration
227
-
*`options.toVersion` (int, optional) - version to which the repo should be migrated to. If left out the version of latest migration is used.
228
-
*`options.ignoreLock` (bool, optional) - if true won't lock the repo for applying the migrations. Use with caution.
229
-
*`options.repoOptions` (object, optional) - options that are passed to migrations, that use them to correctly construct datastore. Options are same like for IPFSRepo.
224
+
*`options.toVersion` (int, optional) - version to which the repo should be migrated. Defaults to the latest migration version.
225
+
*`options.ignoreLock` (bool, optional) - if true will not lock the repo when applying migrations. Use with caution.
226
+
*`options.repoOptions` (object, optional) - options that are passed to migrations, that use them to construct the datastore. (options are the same as for IPFSRepo).
230
227
*`options.onProgress` (function, optional) - callback that is called after finishing execution of each migration to report progress.
231
-
*`options.isDryRun` (bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change.
228
+
*`options.isDryRun` (bool, optional) - flag that indicates if it is a dry run that should give the same output as running a migration but without making any actual changes.
*`migration` (object) - object of migration that just successfully finished running. See [Architecture of migrations](#architecture-of-migrations) for details.
239
-
*`counter` (int) - current number of migration in the planned migrations streak.
240
-
*`totalMigrations` (int) - total count of migrations that are planned to be run.
236
+
*`counter` (int) - index of current migration.
237
+
*`totalMigrations` (int) - total count of migrations that will be run.
Executes backward migration to a specific version.
245
242
246
243
**Arguments:**
247
244
248
245
*`path` (string, mandatory) - path to the repo to be reverted
249
246
*`toVersion` (int, mandatory) - version to which the repo should be reverted to.
250
247
*`options` (object, optional) - options for the reversion
251
-
*`options.ignoreLock` (bool, optional) - if true won't lock the repo for applying the migrations. Use with caution.
252
-
*`options.options` (object, optional) - options that are passed to migrations, that use them to correctly construct datastore. Options are same like for IPFSRepo.
248
+
*`options.ignoreLock` (bool, optional) - if true will not lock the repo when applying migrations. Use with caution.
249
+
*`options.options` (object, optional) - options that are passed to migrations, that use them to construct the datastore. (options are the same as for IPFSRepo).
253
250
*`options.onProgress` (function, optional) - callback that is called after finishing execution of each migration to report progress.
254
-
*`options.isDryRun` (bool, optional) - flag that indicates if it is a dry run that should imitate running migration without actually any change.
251
+
*`options.isDryRun` (bool, optional) - flag that indicates if it is a dry run that should give the same output as running a migration but without making any actual changes.
255
252
256
253
### `getLatestMigrationVersion() -> int`
257
254
258
-
Return the version of latest migration.
255
+
Return the version of the latest migration.
259
256
260
257
## CLI
261
258
262
-
The package comes also with CLI that is exposed as NodeJS binary with name`jsipfs-repo-migrations`.
259
+
The CLI is a NodeJS binary named`jsipfs-repo-migrations`.
263
260
It has several commands:
264
261
265
262
*`migrate` - performs forward/backward migration to specific or latest version.
0 commit comments