JRuby examples don't work (JRuby picks the wrong overloaded method)

[iconara]

The JRuby adapter doesn't work because JRuby can't figure out which overloaded Observable#subscribe method to pick. It ends up picking Observable#subscribe(Map<String, Object>), which raises an error because the argument doesn't respond right:

require 'rxjava-core-0.10.0.jar'
require 'rxjava-jruby-0.10.0.jar'

observable = Java::Rx::Observable.toObservable(['one', 'two', 'three'])
observable.take(2).subscribe(lambda { |x| puts x })

the code above prints the following errors in JRuby 1.7.4:

% ruby rxjava.rb
rxjava.rb:6 warning: ambiguous Java methods found, using subscribe(java.util.Map)
onNext
Observable.java:278:in `subscribe': java.lang.RuntimeException: 'onNext' key must contain an implementation
    from NativeMethodAccessorImpl.java:-2:in `invoke0'
    from NativeMethodAccessorImpl.java:57:in `invoke'
    from DelegatingMethodAccessorImpl.java:43:in `invoke'
    from Method.java:601:in `invoke'
    from JavaMethod.java:455:in `invokeDirectWithExceptionHandling'
    from JavaMethod.java:316:in `invokeDirect'
    from InstanceMethodInvoker.java:61:in `call'
    from MethodHandle.java:599:in `invokeWithArguments'
    from InvocationLinker.java:153:in `invocationFallback'
    from rxjava.rb:5:in `__file__'
    from rxjava.rb:-1:in `load'
    from Ruby.java:807:in `runScript'
    from Ruby.java:800:in `runScript'
    from Ruby.java:669:in `runNormally'
    from Ruby.java:518:in `runFromMain'
    from Main.java:390:in `doRunFromMain'
    from Main.java:279:in `internalRun'
    from Main.java:221:in `run'
    from Main.java:201:in `main'

in JRuby 1.6.8 it prints a less verbose version of the same error.

Notice the line which reads "onNext". That's actually the puts from lambda { |x| puts x } in action. JRuby wraps the lambda in something that looks like a Map, and then when RxJava calls get on that map JRuby calls call on the lambda.

So if you modify the example code to read lambda { |x| lambda { |y| puts y } } this is what happens:

% ruby rxjava.rb
rxjava.rb:6 warning: ambiguous Java methods found, using subscribe(java.util.Map)
RxJava => Could not find function language adaptor: Groovy with path: rx.lang.groovy.GroovyAdaptor
RxJava => Successfully loaded function language adaptor: JRuby with path: rx.lang.jruby.JRubyAdaptor
RxJava => Could not find function language adaptor: Clojure with path: rx.lang.clojure.ClojureAdaptor
RxJava => Could not find function language adaptor: Scala with path: rx.lang.scala.ScalaAdaptor
one
two

Which kind of works, but it's not as smooth as the ideal API would be:

observable.subscribe { |x| puts x }

But that would probably require a real JRuby native extension (if I get the time I'll send you a pull request with one).

[benjchristensen]

Glad to have you involved, we haven't had much use of JRuby (that I'm aware of) so I'm not surprised we have some issues. I would appreciate your involvement.

Can you check out code from this pull request (#319), build the rxjava-jruby jar and try it? We are working on a very different approach in this pull request for language adaptors that is statically typed.

As for the subscribe(Map) overload ... I'm not thrilled by that one and it has caused other issues. I've considered getting rid of it before we hit 1.0, but I know it's used in a variety of places so haven't pulled that trigger.

[iconara]

It works better, but not perfect. JRuby seems to pick a working overload, but it still warns about there being multiple overloads (which is hard to fix without adding JRuby-metadata to the Observable class).

An upside is that you can skip the lambda { ... } and use the more Ruby-ish: obs.subscribe { |x| puts x } (i.e. pass a block to #subscribe).

Since there will be a version generated to work specially for JRuby, how much can the code generation things add? Ideally it would add JRuby metadata to the class to make it work as a JRuby native extension. Can you give me any hints on how to dig into that? Where should I start to look in the code to understand the code generation?

[iconara]

If the generated Observable could be something like this (but lots more, obviously), it could integrate with JRuby even better:

@JRubyClass(name="Rx::Observable")
public class Observable<T> extends RubyObject {
    @JRubyMethod(meta = true, rest = true)
    public static <T> Observable<T> from(ThreadContext ctx, IRubyObject receiver, IRubyObject[] args) {
        // ...
    }

    @JRubyMethod(name = "subscribe")
    public IRubyObject subscribeRuby(ThreadContext ctx, Block block) {
        // ...
    }

There would be some more code needed that registered the class with the runtime, and created the Rx module, and some more things, but that wouldn't need to be generated.

[benjchristensen]

Can you take a look at pull request #323 and suggest how to re-add JRuby support correctly? We have decided against doing byte-code generation.

We can not add anything (such as the annotations in your example) to rxjava-core, it needs to be in a separate submodule, similar to Groovy, Scala and Clojure. So this means using something like extension methods, implicits, or other such tools if JRuby supports them.

[iconara]

I will have a look.

[iconara]

With some small patches at least some of the basics work:

require 'rxjava-core/build/libs/rxjava-0.10.1-SNAPSHOT.jar'

class Java::Rx::Observable
  java_alias :subscribe, :subscribe, [Java::RxUtilFunctions::Action1.java_class]
end

module Rx
  include_package 'rx'
end

o = Rx::Observable.from([1, 2, 3])
o.map { |n| n * 2 }.subscribe { |n| puts(n) }

I haven't tested any more than that, but it looks like it could work.

[jmhofer]

#323 should be 0.11.0-SNAPSHOT afaik, not 0.10.1-SNAPSHOT. Are you sure you're using the correct version of RxJava?

[iconara]

Good catch, I'm not used to gradle, or even having to compile stuff, so I didn't clean up from the last build I did so the old jars were still around.

The Ruby patch works with 0.11.0 too (which means that those small patches would have make the old version work too, so that's good to know). Without the patch it works as with #319, the code runs but it prints a warning about not finding the right overloaded method.

The boring part of doing it this way is that each method on Observable, and any other classes needed for interoperability will need to be annotated with java_alias. On the other hand I don't see any other way of doing it which wouldn't involve writing even more code.

[benjchristensen]

Does JRuby have a way to apply these aliases programmatically? For example, in Groovy we use reflection to determine all methods that need extensions and then programmatically create all of the MetaMethod implementations to bridge Groovy and Java. You can see an example here: https://github.com/mattrjacobs/RxJava/blob/8c87c29bea5e076bdc4202af5626d474ab1c7117/language-adaptors/rxjava-groovy/src/main/java/rx/lang/groovy/RxGroovyExtensionModule.java#L103

Otherwise every time a new method is added to core someone will have to maintain the JRuby java_alias mappings.

[iconara]

Maybe, I'm not sure, I'll have to look into more about how JRuby's native extensions work. So far I've only done it by creating classes and adding metadata, but I guess that it should be possible to do it on existing classes to somehow.

If I understand the code you linked to correctly it's looking through the Observable and BlockingObservable classes for methods that take subclasses of Function and somehow makes the Groovy runtime prefer those over any other overloads. That's what would be needed for JRuby too, and since the Groovy version just uses Java reflection to do it I think it should be possible in JRuby too.

All that the java_alias thing does is that it looks up a Java method by reflection and adds that method into a cache that JRuby will look in before it looks at the Java class, which is kind of similar to what the Groovy extension thing does, I think.

So, yes, probably, maybe, hopefully, but I'm not sure exactly how right now.

[ragalie]

This isn't complete yet, but here's a first pass at an implementation I think will work. @benjchristensen can you take a look and let me know if this is the direction you're looking for? It's largely cribbed from the Groovy implementation.

ragalie@fc0a706

[iconara]

I'm getting warnings about overloads.

This is my code:

require 'rxjava-core/build/libs/rxjava-core-0.14.2-SNAPSHOT.jar'
require 'language-adaptors/build/libs/language-adaptors-0.14.2-SNAPSHOT-sources.jar'

module Rx
  include_package 'rx'
end

o = Rx::Observable.from([1, 2, 3])
o.map { |n| n * 2 }.subscribe { |n| puts(n) }

and this is the output (JRuby 1.7.4):

rxjava.rb:14 warning: ambiguous Java methods found, using subscribe(rx.util.functions.Action1)
2
4
6

[ragalie]

Just to be clear, the commit I posted doesn't work yet. I just want to make sure that the approach is sane before investing more time in it.

[iconara]

Ok. Yeah, it might work.

Not sure how well the generated methods will work, though: they are added on the classes, but call super to call the original method, but I'm not sure that super is actually what you think it is (it might be, I assume you've run the code, and I have not, so I could be completely wrong). In other words, you're adding a method on Observable, and then you call super in that method to call the original method on Observable, but that should call a method on the superclass of Observable, not the original method -- unless JRuby tries to be clever and figures out that your added method is on the eigenclass and super should refer to Observable, but I don't think it does.

And either way, once you call super JRuby still has to look up the right Java method to call, and it doesn't have any more type information to do it, so it will pick one at random and print out the "ambiguous Java methods found" warning. Even if you've wrapped the proc in an Action I'm not sure that JRuby will pick the right overloaded method. In my experience it tries, but it doesn't always succeed in picking the right method.

I think a better way to solve the problem is to use java_alias as in one of my examples above. That way JRuby will always pick the right overload. I'm not sure all the wrapper classes would be needed in that case, because JRuby will wrap a proc in a proxy class that implements whatever interface is required (and since Function and Action interfaces only really have one method, that also just happens to map straight to interface of proc, I think letting JRuby do it makes sense). There might be a slight performance loss from letting JRuby generate proxies instead of having Java classes, but I think it's greatly outweighted by skipping a large part of the work that JRuby otherwise would have done in finding the right overload.

[ragalie]

You're right, the super approach won't work.

I tried the java_alias approach first and the problem I ran into is that it didn't seem to work correctly when a method had multiple signatures with the same number of arguments. For instance, I tested it with RxJava 0.10 which has subscribe(Hash) and subscribe(Action1) (or something like that). I tried just running java_alias on the methods that had Function arguments and it threw an error when I tried to use the Hash argument. I tried running java_alias on all methods, but running it first on the non-function methods and second on the function methods and that didn't work either (threw an error with a Hash argument).

I'm a relative newbie at JRuby, so totally possible I'm doing something wrong. But from my basic understanding I don't understand how java_alias will be a solution if there are two signatures with the same number of arguments and we want to allow access to both of them.