1
- # rustbuild - Bootstrapping Rust
1
+ # Bootstrapping Rust
2
2
3
3
This is an in-progress README which is targeted at helping to explain how Rust
4
4
is bootstrapped and in general some of the technical details of the build
@@ -8,64 +8,20 @@ system.
8
8
> intended to be the primarily used one just yet. The makefiles are currently
9
9
> the ones that are still "guaranteed to work" as much as possible at least.
10
10
11
- ## Using rustbuild
11
+ ## Using the new build system
12
12
13
13
When configuring Rust via ` ./configure ` , pass the following to enable building
14
14
via this build system:
15
15
16
16
```
17
17
./configure --enable-rustbuild
18
- make
19
18
```
20
19
21
- Afterwards the ` Makefile ` which is generated will have a few commands like
22
- ` make check ` , ` make tidy ` , etc. For finer-grained control, the
23
- ` bootstrap.py ` entry point can be used:
24
-
25
- ```
26
- python src/bootstrap/bootstrap.py
27
- ```
28
-
29
- This accepts a number of options like ` --stage ` and ` --step ` which can configure
30
- what's actually being done.
31
-
32
- ## Configuring rustbuild
33
-
34
- There are currently two primary methods for configuring the rustbuild build
35
- system. First, the ` ./configure ` options serialized in ` config.mk ` will be
36
- parsed and read. That is, if any ` ./configure ` options are passed, they'll be
37
- handled naturally.
38
-
39
- Next, rustbuild offers a TOML-based configuration system with a ` config.toml `
40
- file in the same location as ` config.mk ` . An example of this configuration can
41
- be found at ` src/bootstrap/config.toml.example ` , and the configuration file
42
- can also be passed as ` --config path/to/config.toml ` if the build system is
43
- being invoked manually (via the python script).
44
-
45
- ## Build stages
46
-
47
- The rustbuild build system goes through a few phases to actually build the
48
- compiler. What actually happens when you invoke rustbuild is:
49
-
50
- 1 . The entry point script, ` src/bootstrap/bootstrap.py ` is run. This script is
51
- responsible for downloading the stage0 compiler/Cargo binaries, and it then
52
- compiles the build system itself (this folder). Finally, it then invokes the
53
- actual ` boostrap ` binary build system.
54
- 2 . In Rust, ` bootstrap ` will slurp up all configuration, perform a number of
55
- sanity checks (compilers exist for example), and then start building the
56
- stage0 artifacts.
57
- 3 . The stage0 ` cargo ` downloaded earlier is used to build the standard library
58
- and the compiler, and then these binaries are then copied to the ` stage1 `
59
- directory. That compiler is then used to generate the stage1 artifacts which
60
- are then copied to the stage2 directory, and then finally the stage2
61
- artifacts are generated using that compiler.
62
-
63
- The goal of each stage is to (a) leverage Cargo as much as possible and failing
64
- that (b) leverage Rust as much as possible!
20
+ ## ...
65
21
66
22
## Directory Layout
67
23
68
- This build system houses all output under the ` build ` directory, which looks
24
+ This build system houses all output under the ` target ` directory, which looks
69
25
like this:
70
26
71
27
```
@@ -86,12 +42,6 @@ build/
86
42
debug/
87
43
release/
88
44
89
- # Output of the dist-related steps like dist-std, dist-rustc, and dist-docs
90
- dist/
91
-
92
- # Temporary directory used for various input/output as part of various stages
93
- tmp/
94
-
95
45
# Each remaining directory is scoped by the "host" triple of compilation at
96
46
# hand.
97
47
x86_64-unknown-linux-gnu/
@@ -100,8 +50,7 @@ build/
100
50
# folder is under. The exact layout here will likely depend on the platform,
101
51
# and this is also built with CMake so the build system is also likely
102
52
# different.
103
- compiler-rt/
104
- build/
53
+ compiler-rt/build/
105
54
106
55
# Output folder for LLVM if it is compiled for this target
107
56
llvm/
@@ -118,17 +67,6 @@ build/
118
67
share/
119
68
...
120
69
121
- # Output folder for all documentation of this target. This is what's filled
122
- # in whenever the `doc` step is run.
123
- doc/
124
-
125
- # Output for all compiletest-based test suites
126
- test/
127
- run-pass/
128
- compile-fail/
129
- debuginfo/
130
- ...
131
-
132
70
# Location where the stage0 Cargo and Rust compiler are unpacked. This
133
71
# directory is purely an extracted and overlaid tarball of these two (done
134
72
# by the bootstrapy python script). In theory the build system does not
@@ -144,9 +82,7 @@ build/
144
82
# invocation. The build system instruments calling Cargo in the right order
145
83
# with the right variables to ensure these are filled in correctly.
146
84
stageN-std/
147
- stageN-test/
148
85
stageN-rustc/
149
- stageN-tools/
150
86
151
87
# This is a special case of the above directories, **not** filled in via
152
88
# Cargo but rather the build system itself. The stage0 compiler already has
@@ -160,7 +96,7 @@ build/
160
96
# Basically this directory is just a temporary artifact use to configure the
161
97
# stage0 compiler to ensure that the libstd we just built is used to
162
98
# compile the stage1 compiler.
163
- stage0-sysroot /lib/
99
+ stage0-rustc /lib/
164
100
165
101
# These output directories are intended to be standalone working
166
102
# implementations of the compiler (corresponding to each stage). The build
@@ -172,69 +108,3 @@ build/
172
108
stage2/
173
109
stage3/
174
110
```
175
-
176
- ## Cargo projects
177
-
178
- The current build is unfortunately not quite as simple as ` cargo build ` in a
179
- directory, but rather the compiler is split into three different Cargo projects:
180
-
181
- * ` src/rustc/std_shim ` - a project which builds and compiles libstd
182
- * ` src/rustc/test_shim ` - a project which builds and compiles libtest
183
- * ` src/rustc ` - the actual compiler itself
184
-
185
- Each "project" has a corresponding Cargo.lock file with all dependencies, and
186
- this means that building the compiler involves running Cargo three times. The
187
- structure here serves two goals:
188
-
189
- 1 . Facilitating dependencies coming from crates.io. These dependencies don't
190
- depend on ` std ` , so libstd is a separate project compiled ahead of time
191
- before the actual compiler builds.
192
- 2 . Splitting "host artifacts" from "target artifacts". That is, when building
193
- code for an arbitrary target you don't need the entire compiler, but you'll
194
- end up needing libraries like libtest that depend on std but also want to use
195
- crates.io dependencies. Hence, libtest is split out as its own project that
196
- is sequenced after ` std ` but before ` rustc ` . This project is built for all
197
- targets.
198
-
199
- There is some loss in build parallelism here because libtest can be compiled in
200
- parallel with a number of rustc artifacts, but in theory the loss isn't too bad!
201
-
202
- ## Build tools
203
-
204
- We've actually got quite a few tools that we use in the compiler's build system
205
- and for testing. To organize these, each tool is a project in ` src/tools ` with a
206
- corresponding ` Cargo.toml ` . All tools are compiled with Cargo (currently having
207
- independent ` Cargo.lock ` files) and do not currently explicitly depend on the
208
- compiler or standard library. Compiling each tool is sequenced after the
209
- appropriate libstd/libtest/librustc compile above.
210
-
211
- ## Extending rustbuild
212
-
213
- So you'd like to add a feature to the rustbuild build system or just fix a bug.
214
- Great! One of the major motivational factors for moving away from ` make ` is that
215
- Rust is in theory much easier to read, modify, and write. If you find anything
216
- excessively confusing, please open an issue on this and we'll try to get it
217
- documented or simplified pronto.
218
-
219
- First up, you'll probably want to read over the documentation above as that'll
220
- give you a high level overview of what rustbuild is doing. You also probably
221
- want to play around a bit yourself by just getting it up and running before you
222
- dive too much into the actual build system itself.
223
-
224
- After that, each module in rustbuild should have enough documentation to keep
225
- you up and running. Some general areas that you may be interested in modifying
226
- are:
227
-
228
- * Adding a new build tool? Take a look at ` build/step.rs ` for examples of other
229
- tools, as well as ` build/mod.rs ` .
230
- * Adding a new compiler crate? Look no further! Adding crates can be done by
231
- adding a new directory with ` Cargo.toml ` followed by configuring all
232
- ` Cargo.toml ` files accordingly.
233
- * Adding a new dependency from crates.io? We're still working on that, so hold
234
- off on that for now.
235
- * Adding a new configuration option? Take a look at ` build/config.rs ` or perhaps
236
- ` build/flags.rs ` and then modify the build elsewhere to read that option.
237
- * Adding a sanity check? Take a look at ` build/sanity.rs ` .
238
-
239
- If you have any questions feel free to reach out on ` #rust-internals ` on IRC or
240
- open an issue in the bug tracker!
0 commit comments