Problem with Traits not being Matchable

[bblfish]

A fully minimized version of code to duplicate the problem can be found in the comment below by Symon R..

Compiler version

scala 3.2.1 (older too) and nightly 3.3....

Minimized code

The code that does not compile is on this onepage branch of a repo specially for this Dotty issue. There is now only one file to look at: RDF.scala.

The code is duplicated in full here:

import scala.util.Try

trait RDF:
  rdf =>

  type R = rdf.type
  type Node <: Matchable
  type URI <: Node 

  given rops: ROps[R]
end RDF

object RDF:
  type Node[R <: RDF] = R match
    case GetNode[n] => Matchable //n & rNode[R]

  type URI[R <: RDF] <: Node[R] = R match
    case GetURI[u] => u & Node[R] 

  private type GetNode[N] = RDF { type Node = N }
  private type GetURI[U] = RDF { type URI = U }
end RDF

trait ROps[R <: RDF]:
  def mkUri(str: String): Try[RDF.URI[R]]
  def auth(uri: RDF.URI[R]): Try[String]

object TraitTypes:
  trait Node:
    def value: String

  trait Uri extends Node

  def mkUri(u: String): Uri =
    new Uri { def value = u }

object TraitRDF extends RDF:
  import TraitTypes as tz

  override opaque type Node <: Matchable = tz.Node
  override opaque type URI <: Node = tz.Uri

  given rops: ROps[R] with
    override def mkUri(str: String): Try[RDF.URI[R]] = Try(tz.mkUri(str))
    override def auth(uri: RDF.URI[R]): Try[String] =
      Try(java.net.URI.create(uri.value).getAuthority())

end TraitRDF

class Test[R <: RDF](using rops: ROps[R]):
  import rops.given
  lazy val uriT: Try[RDF.URI[R]] = rops.mkUri("https://bblfish.net/#i")
  lazy val x: String = "uri authority=" + uriT.map(u => rops.auth(u))

@main def run =
  val test = Test[TraitRDF.type]
  println(test.x)

Output

The compilation error can be seen in the github actions failing check here.
But I could not get the -explain flag to work with scala-cli.

Running it on the command line one gets the full error explantion. (After setting the env variables
below

export JAVA_HOME=/Library/Java/JavaVirtualMachines/temurin-17.jdk/Contents/Home
export SCALA_HOME=/usr/local/scala/scala3-3.2.1

The following command

sh bin/clean
sh bin/compileScala
-- Error: scala/RDF.scala:44:8 -------------------------------------------------
44 |  given rops: ROps[R] with
   |        ^
   |object creation impossible, since def auth(uri: RDF.URI[R]): util.Try[String] in trait ROps is not defined
   |(Note that
   | parameter RDF.URI[R] in def auth(uri: RDF.URI[R]): util.Try[String] in trait ROps does not match
   | parameter TraitTypes.Uri & Matchable in override def auth(uri: TraitTypes.Uri & Matchable): util.Try[String] in object rops in object TraitRDF
   | )
-- [E038] Declaration Error: scala/RDF.scala:46:17 -----------------------------
46 |    override def auth(uri: RDF.URI[R]): Try[String] =
   |                 ^
   |   method auth has a different signature than the overridden declaration
   |----------------------------------------------------------------------------
   | Explanation (enabled by `-explain`)
   |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   | There must be a non-final field or method with the name auth and the
   | same parameter list in a super class of object rops to override it.
   |
   |   override def auth(uri: TraitTypes.Uri & Matchable): util.Try[String]
   |
   | The super classes of object rops contain the following members
   | named auth:
   |   def auth(uri: RDF.URI[TraitRDF.R]): util.Try[String]
    ----------------------------------------------------------------------------

But the compiler only has a problem when the URI is implemented in terms of a trait, not when it is implemented in terms of a class.

Expectation

The expectation is that it should work with traits as well as with classes.

A fix is to change the trait Uri to a class Uri as in the commit that lead to the onepage-works branch commit as shown in this minimal diff:

The problem is that it is not always possible use classes only and not traits. A major RDF library RDF4J uses traits for its RDF model, such as IRI.

[szymon-rd]

Reproduced on nightly:
https://github.com/szymon-rd/DottyIssue16247/actions/runs/3540769029/jobs/5944219450

@bblfish Could you provide further minimization? It will be easier to address the problem if you provide a single scenario and make this issue self-contained. The best format would be the one that contains:

• Description of the essence of the problem. What scenario does not work, why do you think it should, and what do you perceive as wrong?
• Minimized scala-cli snippets. Preferably ones that should compile but do not. They should be the shortest possible example of the reasoning in the description.
• Then, some additional information that may help in fixing the bug, for example, the fact that when you use classes instead of traits then it works.

The code that you provided is fairly complex. There is a high possibility that there is a way to show the issue with a shorter, less complex snippet, that would make it much easier for us to address this problem. Editing this issue to contain all the information will also help massively, as one can get pretty lost following the previous repositories, issues, and comments in them.

[bblfish]

The code I posted currently stems from the use case I described in lib unification and match types so it has the advantage of describing a realistic situation.

But I agree. I'll try to remove stuff slowly and see how small I can get.
Doing that will take a bit of time though...

[szymon-rd]

To be fair, it is not that valuable to represent a realistic situation. It's best to require as little context as possible. Representing a realistic situation often requires adding additional, unnecessary information. This is a bug report to the compiler, and it's best to stay on the same level of abstraction that the compiler is operating on - the syntactic and semantic structures. You can, of course, provide a rationale on why it is a bug by relating to a real-world problem - for example, argument that the compiler behaves confusingly in a common real-world case. But the problem itself should be kept as simple as possible, and having as narrow context as possible.

[bblfish]

Ok, @szymon-rd I created a new branch minimization and was able to remove quite a lot of code, as shown by the history. commit bblfish/DottyIssue16247@4159ccd removed too much, and commit bblfish/DottyIssue16247@eaf445a put back the minmum for the problem to reappear, as one can see in the github actions.

The problem appears when I move from the functioning

trait RDF:
  rdf =>

  type R = rdf.type
  
  type Node <: Matchable
  type URI <: Node

  given rops: ROps[R]
end RDF

to the code that works when the types are implemented with classes but
not when they are implemented by reference to traits or java interfaces.

trait RDF:
  rdf =>

  type R = rdf.type
  type rNode <: Matchable
  type rURI <: rNode
  type Node <: rNode
  type URI <: Node & rURI

  given rops: ROps[R]
end RDF

I should have saved the intermediate point which also compiled which I think was

trait RDF:
  rdf =>

  type R = rdf.type
  type rURI <: Matchable
  type Node <: Matchable
  type URI <: Node & rURI

  given rops: ROps[R]
end RDF

For some reason that type of hierarchy works ok for classes but not when traits or interfaces are involved.
Note that in each of these cases the both URI and rURI as well as Node and rNode refer to the same
classes/interfaces. Eg for the class based implementation we have

  override opaque type rNode <: Matchable = cz.Node
  override opaque type rURI <: rNode = cz.Uri
  override opaque type Node <: rNode = cz.Node
  override opaque type URI <: Node & rURI = cz.Uri

and for the trait based implementation we have

 override opaque type rNode <: Matchable = rdfscala.TstNode
  override opaque type rURI <: rNode  = rdfscala.URI
  override opaque type Node <: rNode = rdfscala.TstNode
  override opaque type URI <: Node & rURI = rdfscala.URI

Why do I need that?

Well it allows me to use the concept that types are identified by the operations they support to make life of developers using our library easier. Jena and RDF4J don't really check carefully whether a node of type URI in a graph is a relative or absolute URLs, so we can easily allow both. But we would actually like to keep track of the difference.

• Relative urls can take a base and be absolutized, whereas that operation makes no sense for absolute URIs. The inverse one does of relativizing a url to a base, say when POSTing content to a server in LDP, where the server then gives the resource a name.
• We have allowed relative URLs in banana-rdf but that then was very annoying because I could never tell if I had a graph containing them or not. By making these types explicit we can track when we have graphs with relative urls and when we don't. Say if we read-in a graph with relative urls from the web, I know that there can be none if I fold over all the links in the graph and absolutize them. The underlying java objects are the same, but our knowledge has changed after the operation, and we can reflect that by moving from a graph with rURI to a graph containing only URIs.

[szymon-rd]

Thank you for the additional information! This code is still really complex, and further minimization would be welcome. Making the code simpler and shorter will allow us to address this issue faster.

[bblfish]

I have simplified the code again now here in the min2 branch. I removed the no-longer needed java interfaces, and I merged two files so as to reduce them down to 3. I also removed methods that were not playing a role in the tests.

https://github.com/bblfish/DottyIssue16247/blob/min2

Then I made sure that there was the least difference between the Class and Trait based code, which is now very small as can be seen by this opendiff snapshot.

The code is run https://github.com/bblfish/DottyIssue16247/actions/runs/3594012563

[bblfish]

I have now created a new branch min2-fix which sets out to fix the trait based code in two steps. The second commit to that branch involves just changing trait to abstract class which moves the build from broken to fixed. See the one line diff:

bblfish/DottyIssue16247@2b2e935

That should be a helpful way of looking at the problem as it narrows the change to one simple diff.

[szymon-rd]

There is a lot of information in this post right now. Could you put a self-contained summary with the current status of minimisation with the links and all the required information to look into the problem in the issue itself? By that I mean editing it so it only contains the information that is up-to date and required to debug the issue, removing all the outdated information and other background info like history of the problem. It would be very helpful for us, and again thanks for your work minimizing and simplifying the snippets!