RFC: Improve the module system

[kud1ing]

I am not sure what the chances are, but if it could be improved, it should be before 1.0.
Rust's module system has a steep learning/doing curve. I think there are two reasons for this:

1. Importing functionality and importing namespace are two steps: mod and use. This leads to the shadowing rules, that have weird consequences: you need to import a path via use which is not even defined at this point in time, because use needs to before mod. I find this not only unfamiliar but also counterintuitive.
2. The compiled file becomes a crate. In most languages you can iteratively compose a project of files/modules and use/compile those individually. Since use statements are relative to the crate, the meaning changes depending on which file you currently compile. The consequence is that it makes it more difficult than necessary to first write file a.rs, then b.rs (uses a.rs), and then c.rs (uses b.rs) and make them compile individually. Even in Haskell that is easy. In Rust you either have to go back and forth and adjust the use-staments or only put the mod-statements in a dedicated top-level crate (lib.rs or main.rs) which defies encapsulation.

This could be an issue that is fixable with more/better documentation. However, when you need a lot of words/actions to describe/do something in one system which does not need that care in another system, that could be an indication that there is some accidental complexity. I think this complexity is accidental since i am not yet aware of which gains are compensating for this.

[kud1ing]

CC @bstrie because i think i saw you chime in the complains on IRC yesterday

[huonw]

Crates are pretty similar to Haskell's module system. (Admittedly we don't have the tool support to use them like that easily, yet.)

[kud1ing]

They may be similar, but they are harder to use. Maybe mostly due to aspect "1"?

[Matthias247]

+1 on this topic from my side.
I was also a little bit involved in the IRC dicussion yesterday after I had far more issues divding my code into multiple files and modules than in any other language I worked up to now.

The main thing that bothered me is the binding between files and modules -> each new file automatically generates a new module. When you try to divide your code into many files (e.g. one file per object as often done in C++/C#/others and even enforced in Java) you don't get files but even modules. With the additional modules you have to reimport everything and you can no longer access private fields/methods. That is often required when you have some objects that tightly interact. Imho that leads to either very large source code files or you begin to make much more things public than they should be.

As a backwards compatible improvement to this I could imagine the following:

• Create a alternative import to mod xyz; that imports the content of the other file into the current module. In xyz you could then access all private members of other files that belong to the current module. If you don't like the C idea that each file's content can be arbitrarily included then you can use something like the Dart way: In the main file of the module you specify which other files belong to them by using part "otherfile.rs". In all subfiles you specify to which module they belong by starting with part of lib.rs or even explicitely state the module name part of foomodule;. I like the first one better, because it has less ambiguity.

The incompatible way is to do it simply like C#, Java, AS3, ...:

• In each file you start at the root module and describe absolutely into which module the following code should go. Multiple files can oben the same module and describe new types there. If you reference types of other modules, you either specify the path relative to the current module, use an absolute path or create an alias which imports the type into the current module. This approach requires you to think a little bit more about the complete module structure before implementation. But it is tried, proven and well understood by developers. I can also imagine that type resolution in such a system is easier than what Rust currently does.

Besides that I would think that crate local visibility for types would also help to overcome some of the visibility issues that creating new files->modules involves and would really like to see that feature.

[glaebhoerl]

If you ask me, copy the Haskell module system. It's not great but it is a reasonably working subset of what Rust aims to provide.

Please no. Haskell has many great attributes but the module system is emphatically not one of them.

As far as I'm aware Rust's module system is strictly more expressive than Haskell's. Rust decouples the unit of visibility (a module) from the unit of compilation (a crate), which I think is a smooth move. In Haskell the two concepts are tied together. You can recover a Haskell-like system in Rust by basically just never ever using submodules. Put all your definitions at the top level, and if you want to start a new module, start a new crate as well, and they will be compiled separately. So if that's what you want, you can do it today.

[kud1ing]

Please no. Haskell has many great attributes but the module system is emphatically not one of them.

Currently i have the impression that Rust's is even less practical.

As far as I'm aware Rust's module system is strictly more expressive than Haskell's. Rust decouples the unit of visibility (a module) from the unit of compilation (a crate),

What does this solve? Can you give an example of the day-to-day advantages of this? Currently i only see more code and more complexity := cost.

Put all your definitions at the top level

I don't think that we want to encourage putting unrelated code in a giant crate file. Your other solution would make Rust projects have twice the number of files than a comparable project in many other programming languages.

A couple of points:

1. Rust's module system is awesome. If we can make it more convenient to use, I'm all for it, but no semantic changes please.
2. Yes, the mod/use thing is weird. I believe it's just an implementation detail, the parser tries to resolve paths too early. I don't see a reason why that would need to stay that way forever.
3. Haskell projects start 3-4 directories deep because of the Hackage global module namespace, with one extra file per directory for trivial re-exports. Is that something we should try to emulate? How about having to re-export everything and their dog in the module header?
4. I'm not sure what you mean by "which file you currently compile". You compile a crate, not files. foo.rs is just mod foo { ... } factored out. foo/bar.rs is just mod foo { mod bar {...} }. They have no meaning without the crate root. Granted, the docs don't really tell you and it's not obvious without forcing C++-like namespace declarations all over the place, but it is a sane system*. This is also why Rust "files" can have mutually recursive dependencies while you have to manually break them up in Haskell.

*assuming you only declare mod foo; once, and use it everywhere else. It may or may not be a compiler bug to not check for that.

[huonw]

Re aspect 1 and crates: you import a crate with extern mod, and extern mod statements are required to go before use statements, so 1 doesn't apply (unless I'm misinterpreting it).

Re aspect 1 in general: fn foo() { bar() } fn bar() {} is similar: you're calling a function "before" it exists, and I rarely see people complain about that. (admittedly it's not compulsory to put foo before bar like that, unlike use and items).

[bstrie]

I'm not interested in a complete overhaul of the module system at this point. It's just too late in the game.

What I am interested in is making the current system slightly more restrictive than it currently is, in order to reduce the amount of complexity that a user has to keep in their head.

I propose two restrictions:

1. Forbid modules from having the same name as any of their ancestors. I've already proposed this as Forbid modules from having the same name as any of their ancestors #11037. Our pathing rules (both module search paths and file search paths) are already complex enough without having to worry about which foo in foo::foo::foo your current local namespace is referencing.
2. Forbid shadowing of imports. Make it just as much of an error as fn foo(){} fn foo(){} is today (error: duplicate definition of value). If I understand correctly, this behavior would allow us to remove the completely unintuitive use-before-mod rule, as bemoaned by others. And if forbidding shadowing makes globs harder to use, then so be it. Globs are an anti-pattern, and may very well be behind a feature flag for 1.0 anyway.

[bjadamson]

+1 brstrie

I'm new to rust, but I'm having trouble understanding the current module system. The use-before-mod is completely unintuitive (at-least with the current state of the documentation) to me as a new user. Maybe it will be less confusing in the future.

I also agree that Globs should not make it into 1.0, tooling can be built around the compiler to help automatically insert use/extern mod statements.

[bvssvni]

Coming from C#, the Rust module system did not work like I expected, but since Rust is not a verbose class oriented language like C#, the idea of modules becomes more important. I actually like the way a module is located in a specific file by rigid rules. In C#, one has to rely on the IDE to find the declaration of an object. What makes Rust more complicated, is having to put 'mod' and 'use' at top of each file. This becomes a burden when the hierarchy is still in flux.

My idea is to add a new "header" like file named "rust.rh" per folder which one can declare the 'mod' and 'use' for the source files. All source files are compiled as if "rust.rh" is included directly at top of the source, but it can not contain code or declaration of new modules. No explicit declaration is necessary to use "rust.rh". This means getting started is easier, one can simply create a new empty document in an existing project and start typing. The compiler must ignore circular modules pointing to the same file and to do this it needs to strip "rust.rh" for the file it is processing. For future development the "rust.rh" file can contain instructions to flatten all the files in the folder, making them part of the same module. This will make source easier to manage per folder.

My wish is to not having to type 'mod' and 'use' at all per file.

[Matthias247]

That won't work, since a folder contains many modules (as many as files are in it), and each module could be in a completely different position in the module tree, and have different needs for imports.

I agree to others that explicit imports through "use" can get easier in future as soon as there is a decent IDE.
But if I would currently start porting some of code in other languages to Rust and want to preserve at least parts of the visibility restrictions I'm landing at > 10kLoc source files. This is something that really makes me nervous.

[Kimundi]

+1 to @bstrie from me too.
Though I think Rule 1 could be to restrictive, the important thing is that you can always loose those restrictions in a backwards compatible way later one.

Leaving the shadowing rules like they currently are would block that path in the future.

[bstrie]

@Kimundi , while allowing shadowing post-1.0 is theoretically backwards-compatible, it would be impossible to reintroduce use-before-mod in a backwards-compatible fashion. So we'd have to be content with either never allowing shadowing, OR allowing shadowing but not being able to enforce the perceived "order" with use-before-mod, OR breaking compatibility.

[bstrie]

I would also like to caution everyone again against proposing drastic, sweeping changes to the module system at this stage. This is a highly complex topic with far-reaching implications. We've been through a lot of iteration to arrive at our current system, and while it could be better it's also not nearly as bad as it used to be (who else remembers export? or .rc files?). Further iteration upon the current system is what we must strive for.

Remember that there are only three hard problems in programming language design:

1. Module systems
2. Error handling
3. Comment syntax