Skip to content

User Type Highlighting #431

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 312 commits into
base: master
Choose a base branch
from

Conversation

Iron-E
Copy link

@Iron-E Iron-E commented Dec 11, 2020

Closes #21.
Closes #102.
Closes #142.
Closes #205.
Closes #373.
Closes #397.
Closes #428.
Closes #477.


Yesterday I opened a PR and it was closed. Despite this, the reviewer said:

you’re welcome to try making a deliberately more full-featured syntax file.

So I did. Now, types are highlighted in most places rather than just struct definitions. I used active community repositories as a benchmark, such as Ripgrep.

Further, if changes to my chosen names are desired, just ask :). I'm sure we can figure something out.

This PR does a few things of direct relation to #142:

  1. Add rustNoise to allow highlighting of commas, semicolons, dot operators, and other noise characters.
  2. Add rustBounds to allow highlighting of : in where Foo : Bar from a struct definition.
  3. Remove unutilized generics region implementation in favor of rustGenericRegion.
  4. Add syntax highlighting for fold regions using matchgroup.
  5. Add syntax highlighting for all user-defined types through @rustIdentifiers.
  6. Rename types to fit expected naming from other syntax files in Vim community, according to the table below:
    Old Name New Name
    rustIdentifier rustType
    rustType rustBuiltinType
    <variables> rustIdentifier
    <constants> rustConstant
    • See :h group-name for more information.

Topics left to discussion:

  • Default highlight links for new groups?
    • I have some defaults created here but I'm not insistent on imposing them as the defaults.

If you find some place where there wasn't coverage let me know.

master This PR
master PR

@rust-highfive
Copy link

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @da-x (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@Iron-E Iron-E changed the title Feature/struct definition identifiers User Type Highlighting Dec 11, 2020
@Iron-E Iron-E marked this pull request as draft December 11, 2020 03:57
@Iron-E Iron-E marked this pull request as ready for review December 11, 2020 23:19
@Iron-E Iron-E force-pushed the feature/struct-definition-identifiers branch 2 times, most recently from 77ffd41 to a3bb8a5 Compare December 14, 2020 07:36
@da-x
Copy link
Member

da-x commented Dec 14, 2020

Thanks @Iron-E, this is amazing work! Largest update to the syntax description in quite awhile. @chris-morgan please help review this?

There is one specific case when we know that a token is a modpath, but
it doesn't precede a '::'. Example:

```rust
use foo;
```

We know `foo` must be a module, but under normal circumstances it
appears like an Identifier. This commit adds special logic to identify
these cases.
@Iron-E Iron-E force-pushed the feature/struct-definition-identifiers branch from a3bb8a5 to e0babbf Compare December 14, 2020 16:41
Whenever attemping to access an enum:

```rust
Option::Some
Iso::USA
```

`Option` and `Iso` would be highlighted as if they were modules
rather than a types. To fix this, the definition of a `rustModPath`
was altered to contain `@rustIdentifiers`.
There was some repetition of which comment groups were `contained` in
folds, and which weren't. Some even dictated inclusion of nested groups,
which seem to be intended for nesting within a `contained` top-level
comment.

This commit adds a dedicated `cluster` for including the top-level
`rustComment`* groups, and replaces usages where applicable.
Allows highlighting of `&` in syntax such as:

```rust
pub struct Location<'outer>
{
	pub outer: Option<&'outer Self>,
}
```
`pub`lic `type`defs were not being detected.
`let foo = match bar {};` wasn't highlighting `match`.
Makes `dyn` highlighted in:

```rust
&dyn FooTrait
```
Before we did not highlight `as` in a use block, like so:

```rust
use core::fmt::{Display, Result as FmtResult};
```

Now `as` is highlighted properly.
The `pub?` check should have been outside the alternation scope, since
both `use` and `mod` can be `pub`.
Both `mod` and `use` can be `pub`, so the check for whether or not `pub`
was present should not have been in an alternation specific to `mod`.
Previously this group was needed to detect mod paths. Recent updates to
the mod path groups have allowed this to be removed entirely.
Whenever using `return` explicitly, there was an opportunity for either
`match` or `await` to be used after `return`. Despite this, I had
selected `@rustIdentifiers` as the `nextgroup`— this caught all the
keywords and as such `match` and `await` never showed as expected.
@Iron-E
Copy link
Author

Iron-E commented Dec 29, 2020

Sorry for that last round of commits— there were some more issues that I uncovered which needed to be ironed out. Haven't discovered anything else in about a week since though so it's probably close in terms of function.

I will say that there are some things which this implementation doesn't try to do, like tell the difference between typedefs, traits, and the like. There really isn't a great way to know for sure whether or not something is just a "type"/struct or if it's something more specific without leveraging a language server (which I believe is also being discussed).

So, these changes would probably act as a fallback for when the LSP is not available (or as a bridge until the LSP highlighting is integrated).

Iron-E added 5 commits January 5, 2021 01:03
It allows for supertraits with generic bounds that have the following
syntax:

```rust
pub trait CrudOrganization<T> : Into<T> + where WLocation : Wrapper<Location<'name>>, { }
```
Previously, the following `dyn` would have been highlighted as an
identifier:

```rust
pub trait Foo<T> where T : Bar<dyn SomeSizedTrait> {}
```

Now it will be detected as `rustDynKeyword`.
```rust
impl Foo
for Bar
{}
```

The above is valid syntax, but `for` was not being highlighted correctly
in that case.
```rust
($name: ident, $($life: lifetime)*, $T: ident)
```

Now, `$name` and `$life` are highlighted the same. `ident` and
`lifetime` are also highlighted as types, rather than identifiers.
@Iron-E
Copy link
Author

Iron-E commented Apr 27, 2022

I have addressed most of the concerns which you noted in the previous commits. There are only a few questions/comments I had which I am hoping you could clarify/discuss with me further:

As one example, || is normally detected as rustAnonymousFunc or rustOperator correctly, but there’s plenty of perfectly ordinary code that will cause it to fail to detect rustAnonymousFunc, where what was there before just never tried to make the distinction. (Sure, we did already do dubious &amp;&amp; disambiguation, but there at least normal code would never be misidentified.)

The reason why I opted for nextgroup=@rustTokens is that I specifically ordered the syn cluster rustTokens contains= so that 99% of cases would parse correctly without needing further adjustment (since Vim prefers syntax groups earlier in the contains). This is partly because in Rust, as we both know, most tokens can follow most other tokens. In cases where this does not hold, you will see snippets like this:

syn match rustModule "\v(r#)?<\l(\l|\d)*(_+(\l|\d)*)*>" contained contains=rustRawModule,@rustScopes display

As for closures specifically, could you give an example of such breakage? There is a whole section of the sample file dedicated to testing this exact case.

cap

For rustNoise, Delimiter is not appropriate; it’s documented as “character that needs attention”, and in the general group Special; not many syntax files use it for things as pervasive as rustNoise—most leave such characters unhighlighted, which is what I would do here. Yet I personally wouldn’t just do it by not linking rustNoise to anything, I would instead do it with :syn-transparent, because otherwise it loses the highlighting of the parent item, which is important in places like attributes. (Sure, the real problem here is that Vim can’t nest highlighting, e.g. term=inherit,bold cterm=inherit,bold gui=inherit,bold termfg=inherit ctermfg=inherit guifg=inherit to add bold without changing italic or colour, but we work with what we’ve got, and that’s a part of the reason why I changed my attribute highlighting from underlined to italic, because it’s less jarring when some characters inside lose the italicness. As it stands, your non-transparent rustNoise is forcing the parentheses in #[cfg(not(any(unix, other_prop)))] to not be italic, while the rest of the line is.)

This is one area I disagree. The rustNoise characters are delimiters, which Merriam-Webster says are "character[s] that mark the beginning or end of a unit of data". Parentheses, commas, and the like are delimiters in this way. They do need attention to show the delimitation of statements. #[cfg(not(any(unix, other_prop)))] looks how I expect with regard to parens— the only reason #[] doesn't follow the same pattern is that it is a markedly different construct.

I don’t like \v in general because I’m used to reading normal Vim regular expression syntax, and it messes with that, even if it’s by bringing it closer to other regex engine syntaxes. Mixing \v and not-\v is even worse.

\v is closer to Rust's own regex crate, which is why I opted for it. The expressions which do not use \v were either not written by me or were shorter without it (since I am going for readability). :h \v provides a table which makes reading both \v and \V side-by-side — despite being opposites — clear.

In test/sample.rs, you’ve got lots of commas that should be semicolons. (I figure the file should probably parse.)

Could you note an example for this as well?

rustMacroVariable is indiscriminately followed by rustBounds, triggering potentially inappropriate highlighting of stuff after the colon

I don't necessarily think the use of rustBounds is inappropriate. : is conceptually a binding character, which applies broadly from everything to parameters of a function to their types, generics to their bounds, macro variables to a token category, labels to a scope. Can you give an example of such a breakage in this case?

Iron-E added 9 commits May 6, 2022 16:47
The `>` in `'static>` was being treated literally, so there was
mishighlighting of `'static` when inside of a `rustGenericRegion` but
also outside of it.
I thought it would be better to use a different name since this macro
does not panic at runtime, instead it causes compilation to fail.
@mibmo
Copy link

mibmo commented Jul 21, 2022

What's stopping this from getting merged? :)

@Iron-E Iron-E force-pushed the feature/struct-definition-identifiers branch from 1c2592b to d3a27a5 Compare July 25, 2022 19:58
@Iron-E Iron-E mentioned this pull request Jul 26, 2022
@Iron-E
Copy link
Author

Iron-E commented Jul 26, 2022

@chris-morgan or @da-x any updates? It would be good to finally push this through.

Even though this PR isn't as good as something like LSP or Treesitter highlighting— it's still an improvement to and has fewer bugs than master :)

Iron-E added 5 commits July 28, 2022 20:11
It was mislighting certain things, for example:

```rust
macro_rules! foo {
    // Invalid syntax; highlights correctly
    ($($bar:expr),,*) => {},

    // Valid syntax; highlights incorrectly
    ($($bar:expr) *) => {},
}
```

This is now fixed.
@Iron-E
Copy link
Author

Iron-E commented Mar 23, 2023

@chris-morgan @da-x checking in again— what is the status of this pull? Is there anything blocking it?

@da-x
Copy link
Member

da-x commented Mar 24, 2023

@chris-morgan only waiting your ack, being it a big change around highlighting.
I've been using it for awhile and happy.

It did not recognize this repeat delimiter, even though it is valid.
@ModProg
Copy link

ModProg commented May 22, 2023

I feel like the keywords in macro_rules could be actually highlighted maybe (i.e. pub fn self...).
The current branch:
grafik

Master:
grafik

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment