Proposal: Autolink with `[code]`

[wojtekmach]

Currently we autolink expressions like `String`, `String.upcase/1`, etc. The obvious problem is auto linking "Erlang" modules, e.g.: `:file`, which we don't do, because we don't know if the user intended to link to a module or just wrote the atom :file. We can today solve this by writing [`:file`](`:file`) but it's verbose as well as when any other Markdown parser sees it, it will output garbage.

Instead of implicitly treat `code` as potential links, I'd like to propose supporting an explicit syntax: [`code`]. This is what Rust does. See erlef/build-and-packaging-wg#60 (comment). By having explicit syntax, the tooling will know the user intended to write a code reference and thus can output warnings, etc. This will have much more predictable result on non-ExDoc: GitHub, etc would render [`:file`] as [<code>:file</code>] which I think is very reasonable.

Here's a proof of concept: https://gist.github.com/ef18199a4283d9a59b9fdfb571a4e251.

Per erlef/build-and-packaging-wg#60, there are conversations about introducing another EEP that would make Markdown officially supported document format for Erlang. I think there's appetite for supporting the syntax I'm proposing there. If it would be accepted there then I think there'd be a strong reason to support it in Elixir ecosystem too.

[josevalim]

I am fine with supporting this but only after we agree on the format for Erlang so we don't end up with three distinct ways in the future. :)

[Eiji7]

Interesting! How about change it by using sigil-like syntax and modifiers?

For example:

1. default module or mfa (without optional modifier): ~`String` or ~`String.upcase/1` or ~`:file`
2. same module or mfa (with m modifier): ~`String`m or ~`String.upcase/1`m or ~`:file`m
3. Elixir forum topic: ~`53184`f or ~`elixir-v1-14-3-released`f
   Note: both elixirforum.com/t/:id and elixirforum.com/t/:slug redirects to elixirforum.com/t/:slug/:id
   And other forums with for example t modifier for devtalk and e modifier for erlang forums
4. githab links: ~`elixir-lang/ex_doc#1663`g or ~`elixir-lang/ex_doc@4b5c414`g or ~`elixir-lang/ex_doc@tag-v0.29.3`g
5. hex package: ~`phoenix`h
6. hex documentation: ~`phoenix`d or ~`phoenix-1.7.0`d
7. wikipedia article: ~`Elixir_(programming_language)`w or ~`pl.Elixir_(język_programowania)`w

and so on …

[wojtekmach]

These are interesting ideas but remember that one of the goals is whatever we write in the docs should be understandable when rendered by Markdown readers other than ExDoc: IEx, GitHub, etc. In those cases, ~`Elixir_(programming_language)`w is too opaque.

[Eiji7]

I see … nobody said that modifier needs to be a single letter. The rest is about ~ before said link. For me it just look better, but we can do similar for square brackets as well, right?

For example: [wiki:`Elixir_(język_programowania)`:pl]. Maybe even :pl is looks better here, so it's more clear that's an option for wiki modifier and also we do not need to parse it from the beginning of wiki slug.

Edit: Oh, it looks even better for forum modifier example: [forum:`elixir-v1-14-3-released`:elixir]

What do you think?

[wojtekmach]

Sorry, I'm not sold writing [wiki:`Elixir_(język_programowania)`:pl] is better than https://pl.wikipedia.org/wiki/Elixir_(język_programowania). I'd personally choose the latter every time as it's unambiguous and interoperable.

[hauleth]

Wild @hauleth appeared

Maybe, just maybe, we could do something to extend the linking support in general by using custom URIs in classic []() and/or <> MD constructs. So we would support configuration option to the ExDoc generator like :link_schemas that would accept keyword list/map in form of:

%{
  "erlang" => ExDocsErlang.URIHandler
}

ExDocsErlang.URIHandler would then would look like:

def module ExDocsErlang.URIHandler do
  # This is used in case of `<erlang:crypto>` to describe what should be show,
  # defaults to just displaying `foo` or `erlang:crypto`, that is up to the discussion.
  def display_name(value) do
    # Here `value` is `"crypto"`, aka the part of the "URI" with "schema" trimmed
    ":#{value}"
  end

  def url(value) do
    # Here we construct `%URI{}` or just string that will link to the given page
  end
end

By default ExDoc would provide few "built-in" handlers like:

• http and https (either as handlers or as just "magic") - regular HTTP links
• xref - cross page references within the same generation unit
• elixir - link to the Elixir's documentation

So instead of introducing some new magic syntax we will just extend existing one. So current `String` could be written as <xref:String> as well (assuming that we generate documentation for Elixir or <elixir:String> if we generate documentation for other packages and we want to link to official Elixir's documentation. In addition to that users could define their own schemas, which would allow additional features to be added in the future, like for example I see potential for:

• github/gitlab - link to issues and PRs/MRs from the documentation
• livebook - links to the LiveBook documents hosted somewhere else
• wiki - links to Wikipedia or internal Wikis without typing whole URL

I think that some people would even come with creative ways to abuse that for weird purposes, but in general I find it pretty safe and backwards compatible way to support wide range of potential links.

[wojtekmach]

Extensibility via custom schemas is a really nice idea. Similar to previous idea, I don't think it looks good when rendered by GitHub though:

and I personally consider rendering markdown by non-ExDoc tools like GitHub to be crucial.

So instead of introducing some new magic syntax we will just extend existing one.

I'd push back on "new magic syntax" a bit. I'd say it's "implicit". With what I'm proposing, let's say when generating docs for https://hexdocs.pm/elixir,

This is a [`String`].

is just a shortcut for:

This is a [`String`].

[`String`]: https://hexdocs.pm/elixir/String.html

And in the absence of this [`String`]: ..., GitHub would render it as:

and I'd say it's pretty decent but it's subjective.

This is an aside, but to make the point the syntax I propose is pretty natural, I have this in Req README.md:

(...)

  * An easy to use high-level API: [`Req.request/1`], [`Req.new/1`], [`Req.get!/2`], [`Req.post!/2`], etc.

(...)

[`Req.request/1`]: https://hexdocs.pm/req/Req.html#request/1

and this way my README renders nicely on both ExDoc and GitHub:

and if I was able to change the markdown to something like this:

(...)

  * An easy to use high-level API: [`Req.request/1`], [`Req.new/1`], [`Req.get!/2`], [`Req.post!/2`], etc.

(...)

<!-- ex_doc:begin-ignore -->

[`Req.request/1`]: https://hexdocs.pm/req/Req.html#request/1

<!-- ex_doc:end-ignore -->

then I could have cake and eat it too. ExDoc would resolve these links by itself, e.g. [`Req.request/1`] would point to Req.html#request/1, a relative link, which means package versioning will just work. And then non-ExDoc will resolve links too.

[hauleth]

Well, my proposal is to leave "regular" syntax of `String` as is, just add support for some extra stuff in that rare cases when needed. Your proposal is nice, but that has a problem that it uses CommonMark parsing problems. I think that many such "handling" references can cause rendering slowdowns as it will require many additional lookups for the defined links. While <xref:String> do not render nicely I think that in many cases it will not be used, it will be more of an escape hatch. In the same way with my proposal you can do:

[`Req.request/1`](xref:Req.request/1)

If you worry about rendering in the GitHub or other non-ExDoc rendering tool. It will create dangling link, but will render correctly like that:

Req.request/1

_{It seems that GitHub strips unknown schema reference as well}

[wojtekmach]

If it's mostly an escape hatch, what would be some examples for xref: and elixir: links?

It's up in the air still, but if Erlang adopts markdown and this proposal, we could refer to the file module as [`file`] (or [`:file`] from Elixir-style docs) as opposed to, say, [`file`](xref:file) and I'd consider that a win.

[hauleth]

xref is simple:

You have document like a.md and you want to link to section #foo in document b.md or options list #options in module Foo.Bar. Currently there is no way AFAIK to do such cross-document references, especially when you want to rename them via []() syntax.

Same for elixir:

Imagine that you are writing library that redefines |> and you want to link to Elixir documentation for some parts of the document. Or you want to link for example to compatibility guide from your documentation. Obviously you can use fully qualified link, but good shorthand can became quite useful.

[wojtekmach]

[See "foo" section in b docs](b.md#foo) totally works. We even automatically change .md to .html for html and .xhtml for epub. :)

For custom |>, when referring to the OG one, I'd explicitly write `Kernel.|>/2` (or [`Kernel.|>/2`]), that is.)

Or you want to link for example to compatibility guide from your documentation

Yeah, good point!

[garazdawi]

I agree that [`file`] would be a good addition for Erlang markdown docs.

When working with the Markdown convention for Erlang/OTP I imagined that you could write links like this:

Links

When writing documentation in markdown links are automatically found in any
inline code segment that looks like an MFA. For example:

-doc "See `sub/2` for more details"

will create a link to the sub/2 function in the current module if it exists.
You can also use `sub/2` as the link target. For example:

-doc "See [subtract](`sub/2`) for more details"
-doc "See [`sub/2`] for more details"
-doc "See [subtract] for more details

[substract]: `sub/2`
"
-doc "See [subtract][1] for more details

[1]: `sub/2`
"

The above will all result in the same link being created.

https://garazdawi.github.io/otp/system/documentation.html#links

I think that is the same as what you have proposed.