[GR-47923][GR-48496][GR-44824][GR-48447] Backports for Python.

PullRequest: graalpython/2959

Author: timfel

.gitignore

@@ -23,6 +23,7 @@ GRAALPYTHON_UNIT_TESTS.dist
 mx.graalpython/eclipse-launches
 *.json
 !**/resources/*.json
+!**/META-INF/**/*.json
 !graalpython/lib-graalpython/modules/standalone/shared/native-image-resources.json
 !graalpython/lib-graalpython/modules/standalone/shared/native-image-proxy-configuration.json
 language

CHANGELOG.md

@@ -4,9 +4,9 @@ This changelog summarizes major changes between GraalVM versions of the Python
 language runtime. The main focus is on user-observable behavior of the engine.
 
 ## Version 23.1.0
-* Oracle GraalPy distributions (also known as GraalPy Enterprise) are now available under the [GFTC license](https://www.oracle.com/downloads/licenses/graal-free-license.html). The community builds published on Github have been renamed to `graalpy-community-<version>-<os>-<arch>.tar.gz`.
+* Oracle GraalPy distributions (previously known as GraalPy Enterprise) are now available under the [GFTC license](https://www.oracle.com/downloads/licenses/graal-free-license.html). The community builds published on Github have been renamed to `graalpy-community-<version>-<os>-<arch>.tar.gz`.
 * Add support for the sqlite3 module. This allows many packages like `coverage` or `Flask-SQLAlchemy` to work on top of this embedded database.
-* Provide Windows distributions of GraalPy. This is the first preview of Windows support for this feature, and there are limitations, but pure Python packages like Pygal can be installed with `python -m pip --no-cache install pygal`.
+* Provide Windows distributions of GraalPy. This is the first preview of Windows support for GraalPy, and there are limitations, but pure Python packages like Pygal can be installed with `python -m pip --no-cache install pygal`.
 * The GraalPy standalone tool was updated. You can now build single-file executable Python binaries for Linux, Windows, and macOS. The tool can also generate a skeleton Maven project that sets up a polyglot embedding of Python packages into Java.
 
 ## Version 23.0.0

ci.jsonnet

@@ -1 +1 @@
-{ "overlay": "307e5a817ece2f16b7500db068a58c0538cf2101" }
+{ "overlay": "61d2e798a75f4b48457dda2d3b1d3f2ec83dace7" }

docs/user/PythonStandaloneBinaries.md

@@ -9,43 +9,41 @@ permalink: /reference-manual/python/standalone-binaries/
 With GraalPy, you can distribute Python applications or libraries as standalone binaries or JAR files without any external dependencies.
 The [Truffle framework](https://github.com/oracle/graal/tree/master/truffle) on which GraalPy is built, and the [Sulong LLVM runtime](https://github.com/oracle/graal/tree/master/sulong) that GraalPy leverages for managed execution of Python's native extensions enables users to completely virtualize all filesystem accesses of Python programs, including those to the standard library and installed packages.
 
-GraalPy comes with a module that can create standalone binaries or Java project skeletons.
+GraalPy comes with a module that can create standalone binaries or Java-Python polyglot project skeletons.
 The binaries bundle everything into one native executable.
-The Java skeletons are set up with Maven to build and run self-contained JAR files.
-They can also be used to generate a standalone binary from those JAR files later, so Java skeletons offer more flexibility and control over the steps.
+The polyglot skeletons are set up with Maven to to generate a standalone binary for a simple java - python hello world example and can be used as a starting point or inspiration for further java-python polyglot development.
 
 ### Prerequisite
 
-Set `JAVA_HOME` to use a GraalVM distribution.
+A GraalPy distribution.
+TODO - add link
 
-## Creating GraalPy Binaries
+## Creating GraalPy Native Binaries
 
 Suppose there is a simple Python script, _my_script.py_, that does some useful work when run directly.
 To distribute it as a standalone native binary, run the following command:
 
 ```bash
-graalpy -m standalone binary \
+graalpy -m standalone native \
       --module my_script.py \
       --output my_binary
 ```
 
 It generates a standalone _my_binary_ file which includes the Python code, the GraalPy runtime, and the Python standard library in a single, self-contained executable.
-Use `graalpy -m standalone binary --help` for further options.
-
+Use `graalpy -m standalone native --help` for further options.
+ 
 ## Embedding GraalPy in a Java Application
 
-You can distribute the Python script as a JAR file that runs on GraalVM and includes GraalPy.
-To achieve this, run the `java` subcommand of GraalPy's `standalone` module:
+You can also generate a Java-Python polyglot project skeleton.
+To achieve this, run the `polyglot_app` subcommand of GraalPy's `standalone` module:
 
 ```bash
-graalpy -m standalone java \
-      --module my_script.py \
-      --output-directory MyJavaApplication
+graalpy -m standalone polyglot_app --output-directory MyJavaApplication
 ```
 
-It creates a Java project _MyJavaApplication_. It includes a _pom.xml_ file that makes it easy to generate a JAR file or a GraalVM native executable with Maven.
+It creates a Java project _MyJavaApplication_. It includes a _pom.xml_ file that makes it easy to generate a GraalVM native executable with Maven.
 You can open this Maven project with any Java IDE and edit the main class that was created to modify the Python embedding.
-To build the application, either use `mvn -Pjar package` to create a JAR file, or `mvn -Pnative package` to create a GraalVM native executable.
+To build the application, either use `mvn -Pnative package` to create a GraalVM native executable.
 
 Take a look at the generated _pom.xml_ file.
 There are some options to tweak the performance and footprint trade-off.

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/AssignmentTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/AssignmentTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/AtomsTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/AtomsTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,7 +38,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/AwaitAndAsyncTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/AwaitAndAsyncTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/BasicTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/BasicTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,10 +38,11 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import java.io.File;
 
+import com.oracle.graal.python.pegparser.InputType;
 import org.junit.Test;
 
 public class BasicTests extends ParserTestBase {

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/BenchmarkTest.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/BenchmarkTest.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/ClassDefTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/ClassDefTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,8 +39,9 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
+import com.oracle.graal.python.pegparser.InputType;
 import org.junit.Test;
 
 public class ClassDefTests extends ParserTestBase {

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/DefaultStringFactoryImpl.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/DefaultStringFactoryImpl.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2022, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,7 +38,9 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
+
+import com.oracle.graal.python.pegparser.PythonStringFactory;
 
 public class DefaultStringFactoryImpl implements PythonStringFactory<String> {
     @Override

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/DictAndSetTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/DictAndSetTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/ErrorTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/ErrorTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,10 +38,12 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import java.util.EnumSet;
 
+import com.oracle.graal.python.pegparser.FutureFeature;
+import com.oracle.graal.python.pegparser.InputType;
 import org.junit.Test;
 
 /**

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/FuncDefTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/FuncDefTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import java.io.File;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/GeneratorAndCompForTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/GeneratorAndCompForTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,7 +38,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/ImportTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/ImportTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/LambdaInFunctionTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/LambdaInFunctionTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/ListAndSlicingTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/ListAndSlicingTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2019, 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2019, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -39,7 +39,7 @@
  * SOFTWARE.
  */
 
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/NamedExprTest.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/NamedExprTest.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,7 +38,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;
 

graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/NumberTests.java renamed to graalpython/com.oracle.graal.python.pegparser.test/src/com/oracle/graal/python/pegparser/test/NumberTests.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2022, 2023, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -38,7 +38,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
  * SOFTWARE.
  */
-package com.oracle.graal.python.pegparser;
+package com.oracle.graal.python.pegparser.test;
 
 import org.junit.Test;