[GR-45043] Cleanup after migrating to Prism

PullRequest: truffleruby/4142

Author: eregon

CHANGELOG.md

@@ -3,7 +3,7 @@
 New features:
 
 * C/C++ extensions are now compiled using the system toolchain and executed natively instead of using GraalVM LLVM (Sulong). This leads to faster startup, no warmup, better compatibility, smaller distribution and faster installation for C/C++ extensions (#3118, @eregon).
-* Full suport for the Ruby 3.2 and Ruby 3.3 syntax by adopting the [Prism](https://github.com/ruby/prism) parser (#3117, #3038, #3039, @andrykonchin, @eregon).
+* Full suport for the Ruby 3.2 and Ruby 3.3 syntax by adopting the [Prism](https://github.com/ruby/prism) parser, which is about twice as fast as the old parser (#3117, #3038, #3039, @andrykonchin, @eregon).
 * Pattern matching is now fully supported, with the exception of Find pattern (`in [*, a, *]`) (#3332, #2683, @eregon, @razetime).
 
 Bug fixes:

doc/contributor/how-to-guide.md

@@ -1139,24 +1139,6 @@ jt -q ruby -e 'puts Truffle::Debug.java_class_of([])'
 RubyArray
 ```
 
-### Truffle::Debug.yarp_serialize
-
-```
-jt -q ruby -e 'puts Truffle::Debug.yarp_serialize("1").dump'
-"YARP\x00\x04\x00@7\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00L\f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00S\x19\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00&\b\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00"
-```
-
-### Truffle::Debug.yarp_parse
-
-```
-jt -q ruby -e 'puts Truffle::Debug.yarp_parse("[].to_ary")'
-ProgramNode
-  Scope
-  StatementsNode
-    CallNode
-      ArrayNode
-```
-
 ### Truffle::Debug.ast
 
 ```

doc/contributor/interop_details.md

@@ -78,6 +78,28 @@
 
 # Behavior of interop messages for Ruby objects
 
+## toDisplayString()
+
+When interop message `toDisplayString` is sent
+- to 
+  it returns a String representing the object.
+- otherwise
+  it returns the same as `subject.inspect`.
+
+## MetaObject related messages
+
+When interop message `hasMetaObject` is sent
+- to **`nil`**, **`:symbol`**, **a `String`**, **a `BigDecimal`**, **an `Object`**, **a frozen `Object`**, **a `StructWithValue`**, **a `Class`**, **a `Hash`**, **an `Array`**, **an `Exception`**, **an `Exception` with a cause**, **`proc {...}`**, **`lambda {...}`**, **a `Method`**, **a `Truffle::FFI::Pointer`**, **polyglot pointer**, **polyglot members**, **polyglot array** or **polyglot hash**
+  it returns true.
+- otherwise
+  it returns false.
+
+When interop message `getMetaObject` is sent
+- to 
+  it returns the Ruby exception's class.
+- otherwise
+  it returns the same Ruby class as `subject.class`.
+
 ## `null` related messages
 
 When interop message `isNull` is sent
@@ -329,7 +351,7 @@ When interop message `writeMember` is sent
   it writes the given value under the given name.
 - to **a `StructWithValue`**
   it writes the value to the given struct member.
-- to **a `BigDecimal`**, **`nil`**, **`:symbol`** or **a frozen `Object`**
+- to **`nil`**, **`false`**, **`true`**, **`:symbol`**, **an `Integer`**, **a `Float`**, **a `BigDecimal`** or **a frozen `Object`**
   it fails with `UnsupportedMessageError` when the receiver is frozen.
 - otherwise
   it fails with `UnsupportedMessageError`.

doc/contributor/parser.md

@@ -8,14 +8,74 @@
 
 ## Print Detailed Prism AST
 
+### From the Prism repository
+
 ```bash
-cd prism
-bundle exec rake
-bin/parse -e '1&.itself'
+$ cd prism
+$ bundle exec rake compile
+$ bin/parse -e '1&.itself'
+@ ProgramNode (location: (1,0)-(1,9))
+├── locals: []
+└── statements:
+    @ StatementsNode (location: (1,0)-(1,9))
+    └── body: (length: 1)
+        └── @ CallNode (location: (1,0)-(1,9))
+            ├── flags: safe_navigation
+            ├── receiver:
+            │   @ IntegerNode (location: (1,0)-(1,1))
+            │   └── flags: decimal
+            ├── call_operator_loc: (1,1)-(1,3) = "&."
+            ├── name: :itself
+            ├── message_loc: (1,3)-(1,9) = "itself"
+            ├── opening_loc: ∅
+            ├── arguments: ∅
+            ├── closing_loc: ∅
+            └── block: ∅
 ```
 
+### From TruffleRuby
+
 We can also see what the AST as Java nodes and without extra location fields looks like on TruffleRuby with:
 ```bash
-cd truffleruby
-jt -q ruby -e 'puts Truffle::Debug.yarp_parse(ARGV[0])' -- '1&.itself'
+$ cd truffleruby
+$ jt -q ruby tool/parse_ast.rb '1&.itself'
+Source:
+1&.itself
+
+AST:
+ProgramNode
+  locals:
+  statements: StatementsNode
+    body:
+      CallNode[Li]
+        flags: 1
+        receiver: IntegerNode
+          flags: 2
+        name: "itself"
+        arguments: null
+        block: null
+```
+
+```bash
+$ jt -q ruby tool/parse_ast.rb some_file.rb
+```
+
+You can also compare to JRuby's AST with:
+```bash
+$ jruby tool/parse_ast.rb '1&.itself'
+Source:
+1&.itself
+
+AST:
+RootNode line: 0
+  CallNode*[lazy]:itself line: 0
+    FixnumNode line: 0, long: 1
+, null, null
+```
+
+## Print the Truffle AST
+
+```bash
+$ cd truffleruby
+$ jt -q ruby tool/truffle_ast.rb '1&.itself'
 ```

doc/contributor/prism.md

@@ -321,9 +321,3 @@ file to fix it.
 
 You can also recompute the tags automatically for an entire test file with
 `jt retag test/mri/tests/file.rb`.
-
-## Building the parser
-
-TruffleRuby uses the Jay parser generator. A copy of this is located in
-`tool/jay`. The command `jt build parser` will build Jay, if needed, and then
-regenerate the parser. We check the generated parser into the source repository.

doc/contributor/updating-jruby-parser.md

@@ -208,14 +208,6 @@ Engine Yard and are released under an MIT licence (see `mit.txt`).
 We do not distribute the FFI Specs, but they are copyright 2008-2014
 Ruby-FFI contributors and are released under an MIT licence (see `mit.txt`).
 
-## Jay
-
-TruffleRuby uses the Jay parser generator, modified from
-https://github.com/jruby/jay revision `9ffc59a`. Jay is copyright 1989 The
-Regents of the University of California, ported by Axel T. Schreiner, and is
-covered by the three-clause BSD licence (see `jay.txt`). We only distribute Jay
-in the source repository - it isn't part of releases.
-
 ## Written offer for source code
 
 For any software that you receive from Oracle in binary form which is licensed

doc/contributor/workflow.md

@@ -14,6 +14,41 @@ Also see [Tools](tools.md) for more tools besides just debuggers.
 
 ## VSCode
 
+### Simple and Fast
+
+1. Open VSCode in the project you want to debug:
+    ```bash
+    $ cd /path/to/project
+    $ code .
+    ```
+
+1. Create a file `.vscode/launch.json` with this contents:
+
+```json
+{
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Attach",
+            "type": "node",
+            "request": "attach",
+            "debugServer": 4711
+        }
+    ]
+}
+```
+
+3. Run TruffleRuby on the command line and pass `--dap`, e.g.
+    ```bash
+    $ ruby --dap test.rb
+    $ ruby --dap -Ilib test/some_test.rb
+    $ TRUFFLERUBYOPT=--dap bundle exec rspec some_spec.rb
+    ```
+
+4. In VSCode click on `Run` -> `Start Debugging`.
+
+### With the GraalVM VSCode extension
+
 First install [the GraalVM VSCode extension](https://marketplace.visualstudio.com/items?itemName=oracle-labs-graalvm.graalvm).
 
 Then follow [this documentation](https://www.graalvm.org/latest/tools/vscode/graalvm-extension/polyglot-runtime/#debugging-ruby) to debug TruffleRuby with VSCode.

doc/legal/legal.md

@@ -270,6 +270,10 @@ class EvalSpecs
     ScratchPad.clear
   end
 
+  it "returns nil if given an empty string" do
+    eval("").should == nil
+  end
+
   # See language/magic_comment_spec.rb for more magic comments specs
   describe "with a magic encoding comment" do
     it "uses the magic comment encoding for the encoding of literal strings" do

doc/user/debugging.md

@@ -231,13 +231,23 @@ def spec_it(subject)
       strange_symbol: Subject.(:"strange -=@\0x2397"),
       empty_string:   Subject.() { "" },
       string:         Subject.(name: AN_INSTANCE, doc: true) { "string" },
+      frozen_string:  Subject.() { "frozen".freeze },
 
       zero:          Subject.(0),
       small_integer: Subject.(1, name: AN_INSTANCE, doc: true),
+      long_integer:  Subject.(1 << 60),
+      big_integer:   Subject.(1 << 80),
       zero_float:    Subject.(0.0),
       small_float:   Subject.(1.0, name: AN_INSTANCE, doc: true),
       big_decimal:   Subject.(BigDecimal('1e99'), name: AN_INSTANCE, doc: true),
 
+      # Other immutables
+      range:         Subject.(1..),
+      int_range:     Subject.(1..2),
+      long_range:    Subject.((1 << 60)..(1 << 61)),
+      encoding:      Subject.(Encoding::UTF_8),
+      regexp:        Subject.(/regexp/),
+
       object:        Subject.(name: AN_INSTANCE, doc: true) { Object.new },
       frozen_object: Subject.(name: "a frozen `Object`", doc: true) { Object.new.freeze },
       struct:        Subject.(name: AN_INSTANCE, doc: true, explanation: "a `Struct` with one property named `value`") { StructWithValue.new DEFAULT },
@@ -312,9 +322,9 @@ def spec_it(subject)
           interop_library_reference) { TruffleInteropSpecs::PolyglotHash.new }
   }.each { |key, subject| subject.key = key }
 
-  immediate_subjects     = [:false, :true, :zero, :small_integer, :zero_float, :small_float]
+  immediate_subjects     = [:false, :true, :zero, :small_integer, :long_integer, :zero_float, :small_float]
   non_immediate_subjects = SUBJECTS.keys - immediate_subjects
-  frozen_subjects        = [:big_decimal, :nil, :symbol, :strange_symbol, :frozen_object]
+  frozen_subjects        = SUBJECTS.each_pair.select { |key, subject| key != :raise_exception && subject.value.frozen? }.map(&:first)
   exception_subjects     = [:exception, :exception_with_cause, :raise_exception]
 
   # not part of the standard matrix, not considered in last rest case
@@ -352,6 +362,28 @@ def array_element_predicate(message, predicate, insert_on_true_case)
   end
 
   MESSAGES = [
+      Delimiter["toDisplayString()"],
+      Message[:toDisplayString,
+              Test.new("returns a String representing the object", :strange_symbol, :raise_exception) do |subject|
+                Truffle::Interop.should.string? Truffle::Interop.to_display_string(subject)
+              end,
+              Test.new("returns the same as `subject.inspect`") do |subject|
+                Truffle::Interop.to_display_string(subject).should == subject.inspect
+              end],
+
+      Delimiter["MetaObject related messages"],
+      Message[:hasMetaObject,
+              Test.new("returns true", *non_immediate_subjects, &predicate(:has_meta_object?, true)),
+              Test.new("returns false", &predicate(:has_meta_object?, false))],
+      Message[:getMetaObject,
+              Test.new("returns the Ruby exception's class", :raise_exception) do |subject|
+                Truffle::Interop.meta_object(subject).should == RuntimeError
+              end,
+              Test.new("returns the same Ruby class as `subject.class`") do |subject|
+                # It works for immediate_subjects too because of Truffle::Interop.meta_object handling that
+                Truffle::Interop.meta_object(subject).should == subject.class
+              end],
+
       Delimiter["`null` related messages"],
       Message[:isNull,
               Test.new("returns true", :nil, &predicate(:null?, true)),

spec/ruby/core/kernel/eval_spec.rb

@@ -29,6 +29,16 @@
     Truffle::Interop.meta_object([1, 2, 3]).should == Array
   end
 
+  it "returns Range class for a range" do
+    Truffle::Interop.should.has_meta_object?(1..)
+    Truffle::Interop.should.has_meta_object?(1..2)
+    Truffle::Interop.should.has_meta_object?((1<<60)..(1<<61))
+
+    Truffle::Interop.meta_object(1..).should == Range
+    Truffle::Interop.meta_object(1..2).should == Range
+    Truffle::Interop.meta_object((1<<60)..(1<<61)).should == Range
+  end
+
   it "returns a Ruby class implementing all meta objects methods" do
     meta = Truffle::Interop.meta_object("string")
     Truffle::Interop.meta_simple_name(meta).should == 'String'