Skip to content

source_span_for_markdown_range: fix utf8 violation #141666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

source_span_for_markdown_range: fix utf8 violation #141666

wants to merge 3 commits into from
+54 −36

Conversation

lolbinarycat
Copy link
Contributor

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 27, 2025
@GuillaumeGomez
Copy link
Member

We need a regression test.

@lolbinarycat
Copy link
Contributor Author

yes, I was under the impression you were working on that (well, a MCVE for the issue). if you want me to do it, i can.

@GuillaumeGomez
Copy link
Member

I triggered the issue with sysinfo when running clippy. And I was planning on doing that tomorrow, I'm supposed to sleep right now. ^^'

@lolbinarycat
Copy link
Contributor Author

Alright, I think I should be able to throw something together.

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@lolbinarycat
Copy link
Contributor Author

my initial ideas for how to trigger this crash were unsuccessful, might be easier to just throw treereduce-rustc at it or something.

also not sure if you had the bug happen on unmodified sysinfo or if there was some change you made.

@fmease
Copy link
Member

fmease commented May 27, 2025

See #141665 (comment) for a minimal repro involving clippy-driver. Fn check_attrs in src/tools/clippy/clippy_lints/src/doc/mod.rs does a bunch of pre- and re-processing before/while using the methods exposed in rustc_resolve::rustdoc, so y'all need to figure out if this can even repro under rustc/rustdoc.

@fmease fmease added T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 28, 2025
@lolbinarycat lolbinarycat force-pushed the rustdoc-source_span_for_markdown_range-bug-141665 branch from 3194bc1 to 74b1eb3 Compare May 29, 2025 18:56
@lolbinarycat
Copy link
Contributor Author

@fmease do you think we could just add a regression test to the clippy test suite? either as a followup PR or via submodules.

@GuillaumeGomez
Copy link
Member

Isn't it possible to create a span in a unit test and then run our function on it?

@lolbinarycat
Copy link
Contributor Author

A unit test would almost certainly work, but I'm not sure the proper way to get a TyCtx for testing.

GuillaumeGomez
GuillaumeGomez reviewed May 29, 2025
View reviewed changes
compiler/rustc_resolve/src/rustdoc.rs
@@ -570,7 +570,12 @@ pub fn source_span_for_markdown_range(
{
// If there is either a match in a previous fragment, or
// multiple matches in this fragment, there is ambiguity.
if match_data.is_none() && !snippet[match_start + 1..].contains(pat) {
// the snippet cannot be zero-sized,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please don't split your comments so much. It makes much harder to read. You have up to 100 columns, make use of it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to use semantic line breaks, but maybe i went a bit overboard. I also am using a laptop with a fairly small screen.

@GuillaumeGomez
Copy link
Member

That's a good point. I suppose there are such unit tests in the compiler making use of TyCtxt internally? Maybe try to check that.

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rust-log-analyzer
Copy link
Collaborator

The job aarch64-gnu-llvm-19-1 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot) 
failures:

---- [ui] tests/rustdoc-ui/lints/bare-urls.rs stdout ----

error: /checkout/tests/rustdoc-ui/lints/bare-urls.rs:45: unexpected ERROR: '45:1: 47:34: this URL is not a hyperlink [rustdoc::bare_urls]'

error: /checkout/tests/rustdoc-ui/lints/bare-urls.rs:47: expected ERROR not found: this URL is not a hyperlink

error: 1 unexpected errors found, 1 expected errors not found
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustdoc" "/checkout/tests/rustdoc-ui/lints/bare-urls.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2" "--target=aarch64-unknown-linux-gnu" "--check-cfg" "cfg(test,FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-o" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/rustdoc-ui/lints/bare-urls" "-A" "internal_features" "-Cdebuginfo=0"

--- unexpected errors (from JSON output) ---
ERRORline  45: 45:1: 47:34: this URL is not a hyperlink [rustdoc::bare_urls]
---
--- not found errors (from test file) ---
ERRORline  47: this URL is not a hyperlink
---


thread '[ui] tests/rustdoc-ui/lints/bare-urls.rs' panicked at src/tools/compiletest/src/runtest.rs:790:13:
errors differ from expected

@lolbinarycat
Copy link
Contributor Author

The new code should have exactly identical behavior to the old code, but without the crash.

I thought I fixed the inconsistentcy, but the tests are still failing, which means something changed which shouldn't have.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@GuillaumeGomez GuillaumeGomez GuillaumeGomez left review comments

Assignees

@GuillaumeGomez GuillaumeGomez

Labels
S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Panic in source_span_for_markdown_range
5 participants
@lolbinarycat @GuillaumeGomez @rust-log-analyzer @fmease @rustbot