Skip to content

Port sourcecode to Dotty #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Dec 15, 2019

Conversation

anatoliykmetyuk
Copy link
Collaborator

Here's a short summary of the changes:

  • Do not publish doc jars for Dotty – see Add os-lib to the community build scala/scala3#7588 (comment). @lihaoyi do you think it will prevent us from publishing to Maven so that early Dotty adopters can also use your libraries? Can we address this issue after all the major libs are ported?
  • Add Mill downloader script (copied from utest), use it from Travis.
  • Macro port strategy: for all sources not shared between Scala 2 and Dotty, have Dotty sources stored in the src-0 folder and Scala 2 sources – in src-2. Scala 2 sources are moved without changes.
  • Dotty changes textual representation of certain things – hence the tests needed to be tweaked to account for that.
  • Args does not work so far – see Update Sourcecode to the latest Upstream and Dotty versions scala/scala3#7553 (comment)

@anatoliykmetyuk anatoliykmetyuk marked this pull request as ready for review November 26, 2019 14:44
@lihaoyi
Copy link
Member

lihaoyi commented Nov 27, 2019

Thanks! I'll take a look. @alexarchambault @olafurpg as committers your review would also be very welcome

Copy link
Member

@lihaoyi lihaoyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Looks good!

Args not working is probably fine I think. I never found it work reliably even in Scala 2.x, and it seemed to have a propensity for making the compiler think there's an invalid recursive reference.

I don't think I'm qualified to review sourcecode/src-0/sourcecode/Macros.scala; from a cursory view, it looks like a mostly syntactic rewrite from the original scala-reflect implementation. Hopefully someone more familiar with dotty-reflect can sign off on it!

The overall project layout changes look fine to me.

Only thing I want is getting FullName and Enclosing to match their output in Scala 2; I assume Dotty's macro API provides all the functionality necessary, and you just need to wrestle/mangle it to give the same result. After that I'll be happy to merge and keep this running in CI

import ctx.tasty.given
val owner = actualOwner(ctx.tasty)(ctx.tasty.rootContext.owner)
val simpleName = Util.getName(ctx.tasty)(owner)
'{Name(${simpleName.toExpr})}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the reference to free terms like Name in these splices hygienic? We use the ${c.prefix} incantation in the Scala 2.x implementation to ensure that, not sure if somethign similar is required in Dotty

Copy link
Collaborator Author

@anatoliykmetyuk anatoliykmetyuk Nov 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as my understanding of hygiene in Scala 2 macros goes, in Scala 3 it is fine. Scala 3 quotes are compiled on definition site wheres Scala 2 quasiquotes are compiled on expansion site. So in Scala 2, if e.g. you use Name instead of ${c.prefix} and comment out package sourcecode in Apply.scala tests, you'll get a bunch of errors saying that Name is not found. Whereas it works with the package sourcecode uncommented. In Scala 3, the context of expansion is irrelevant since the quote is already compiled. @nicolasstucki maybe you can provide more precise info on this.

@anatoliykmetyuk
Copy link
Collaborator Author

I don't think I'm qualified to review sourcecode/src-0/sourcecode/Macros.scala; from a cursory view, it looks like a mostly syntactic rewrite from the original scala-reflect implementation. Hopefully someone more familiar with dotty-reflect can sign off on it!

@nicolasstucki @liufengyun if you find the time for this one, your opinion would be much appreciated!

Copy link
Contributor

@liufengyun liufengyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, LGTM

@anatoliykmetyuk
Copy link
Collaborator Author

I've addressed the changes mentioned – WDYT @lihaoyi?
Thank you for finding the time to review this @liufengyun!

@lihaoyi
Copy link
Member

lihaoyi commented Dec 13, 2019

@anatoliykmetyuk I think it looks good. How would you like to proceed from here: would you like to merge this into master and keep it running in uTest's CI? Or would you like to port a few more libraries to Dotty and get them all working together before committing? Or do you want to wait for the next Dotty stable version before publishing these libraries?

I'm fine with whatever, so it would be up to you how you'd like to do this

@anatoliykmetyuk
Copy link
Collaborator Author

I think it's best to keep things as decoupled as possible while aiming at making them available to the end-user fast. What I have in mind is the following.

  • Merge the PRs which use the Dotty nightly build. If a library depends on another library built against Dotty nightly, we'll have to publish that dependency to Maven to make the merge possible.
  • On Dotty RC releases, we submit an update PR bumping the Dotty version for all of your libraries which are ported to Dotty. At this point, it would be nice to publish the libraries built against the Dotty RC release to Maven so that they are available to the users.
  • All of your libraries that are ported to Dotty are also included in our community build. If we break some of them and fix them in the community build, we'll also submit an upstream PR built against the latest Dotty nightly version.

This should make sure the changes induced by the Dotty development propagate to the main repo as fast as possible. Also, this ensures that the libraries become available to the community at the same time with the new Dotty RC versions. The relatively frequent changes forced by the Dotty development (especially macros) can be included in the main repos without the need for the library version change since the language version (nightly and RC) will be different for these changes.

The only possible drawback here is the point of releasing the libraries to Maven on each Dotty RC build and when a library depends on a nightly version of another library. We release every 6 weeks, and if your CI doesn't automatically publish to Maven, this scenario may tax you with the work needed to publishing the libraries every 6 weeks.

What do you think?

@lihaoyi
Copy link
Member

lihaoyi commented Dec 14, 2019

@anatoliykmetyuk that plan sounds reasonable to me. One caveat is that since Dotty releases relatively frequently, we should publish versions of these libraries for new versions of Dotty without incrementing the library's version number. That should be doable simple by selecting the dotty cross builds when we run publishAll.

I can give you my publishing credentials, that way you'll have full flexibility to publish any and all of these libraries for Dotty as and when you see fit: whether every 6 weeks, or even more frequently. I can also give you commit bit on these repos, so for minor changes that do not require a code review (e.g. tweaks to the Dotty-specific code) you can simply make them without needing to go the latency of a PR review & merge (though for any large changes it's still best to get reviewed)

If we're OK with that, I suppose the next step would be merging this PR, and giving you publishing/commit access. Let me know and i'll send you the invite/creds

@anatoliykmetyuk
Copy link
Collaborator Author

Sounds good to me – thanks for making this happen!

One caveat is that since Dotty releases relatively frequently, we should publish versions of these libraries for new versions of Dotty without incrementing the library's version number. That should be doable simple by selecting the dotty cross builds when we run publishAll.

Yes, the Dotty-induced changes will be tagged with the language version, so we don't need to change the library version.

@lihaoyi
Copy link
Member

lihaoyi commented Dec 15, 2019

Thanks @anatoliykmetyuk, I have sent you an invite for this repo and emailed you the publishing creds. You may merge when ready

@anatoliykmetyuk anatoliykmetyuk merged commit 768718c into com-lihaoyi:master Dec 15, 2019
@anatoliykmetyuk
Copy link
Collaborator Author

Merged it. The next Dotty release is the following week, so I'll try to publish the Dotty version to Maven then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants