Skip to content

Commit ac3d8ce

Browse files
Moving polyglot java import handling into EpbLanguage (#13483)
1 parent 209a7cb commit ac3d8ce

File tree

40 files changed

+1687
-797
lines changed

40 files changed

+1687
-797
lines changed

build.sbt

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2131,6 +2131,8 @@ lazy val `language-server` = (project in file("engine/language-server"))
21312131
(`language-server-deps-wrapper` / Compile / exportedModule).value,
21322132
(`fansi-wrapper` / Compile / exportedModule).value,
21332133
(`text-buffer` / Compile / exportedModule).value,
2134+
(`jvm-channel` / Compile / exportedModule).value,
2135+
(`jvm-interop` / Compile / exportedModule).value,
21342136
(`runtime-suggestions` / Compile / exportedModule).value,
21352137
(`runtime-parser` / Compile / exportedModule).value,
21362138
(`runtime-compiler` / Compile / exportedModule).value,
@@ -2415,10 +2417,13 @@ lazy val `runtime-language-epb` =
24152417
"org.graalvm.sdk" % "nativeimage" % graalMavenPackagesVersion
24162418
),
24172419
Compile / internalModuleDependencies := Seq(
2420+
(`jvm-channel` / Compile / exportedModule).value,
2421+
(`jvm-interop` / Compile / exportedModule).value,
24182422
(`ydoc-polyfill` / Compile / exportedModule).value,
24192423
(`runtime-utils` / Compile / exportedModule).value
24202424
)
24212425
)
2426+
.dependsOn(`jvm-interop` % Test)
24222427

24232428
lazy val `runtime-language-arrow` =
24242429
(project in file("engine/runtime-language-arrow"))
@@ -2697,6 +2702,8 @@ lazy val `runtime-integration-tests` =
26972702
(`connected-lock-manager` / Compile / exportedModule).value,
26982703
(`library-manager` / Compile / exportedModule).value,
26992704
(`persistance` / Compile / exportedModule).value,
2705+
(`jvm-channel` / Compile / exportedModule).value,
2706+
(`jvm-interop` / Compile / exportedModule).value,
27002707
(`interpreter-dsl` / Compile / exportedModule).value,
27012708
(`engine-common` / Compile / exportedModule).value,
27022709
(`edition-updater` / Compile / exportedModule).value,
@@ -2883,6 +2890,8 @@ lazy val `runtime-benchmarks` =
28832890
(`library-manager` / Compile / exportedModule).value,
28842891
(`persistance` / Compile / exportedModule).value,
28852892
(`interpreter-dsl` / Compile / exportedModule).value,
2893+
(`jvm-channel` / Compile / exportedModule).value,
2894+
(`jvm-interop` / Compile / exportedModule).value,
28862895
(`engine-common` / Compile / exportedModule).value,
28872896
(`edition-updater` / Compile / exportedModule).value,
28882897
(`editions` / Compile / exportedModule).value,
@@ -3992,22 +4001,19 @@ lazy val `jvm-interop` =
39924001
(Test / fork) := true,
39934002
commands += WithDebugCommand.withDebug,
39944003
libraryDependencies ++= Seq(
3995-
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
3996-
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided",
3997-
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
3998-
"org.graalvm.sdk" % "graal-sdk" % graalMavenPackagesVersion % Test,
3999-
"junit" % "junit" % junitVersion % Test,
4000-
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
4004+
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
4005+
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided",
4006+
"org.graalvm.sdk" % "graal-sdk" % graalMavenPackagesVersion % Test,
4007+
"junit" % "junit" % junitVersion % Test,
4008+
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
40014009
),
40024010
Compile / moduleDependencies ++= Seq(
4003-
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion,
4004-
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion,
4005-
"org.graalvm.sdk" % "nativeimage" % graalMavenPackagesVersion,
4006-
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion,
4007-
"org.graalvm.sdk" % "word" % graalMavenPackagesVersion
4011+
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion,
4012+
"org.graalvm.sdk" % "nativeimage" % graalMavenPackagesVersion,
4013+
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion,
4014+
"org.graalvm.sdk" % "word" % graalMavenPackagesVersion
40084015
),
40094016
Compile / internalModuleDependencies ++= Seq(
4010-
(`engine-common` / Compile / exportedModule).value,
40114017
(`jvm-channel` / Compile / exportedModule).value,
40124018
(`persistance` / Compile / exportedModule).value
40134019
)

docs/polyglot/README.md

Lines changed: 35 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,21 +8,46 @@ order: 0
88

99
# Enso Polyglot Support
1010

11-
Enso supports robust polyglot interoperation with other programming languages
12-
that are supported on its platform. This section of the design documentation
13-
deals with
11+
Enso supports robust interoperability with other programming languages - e.g.
12+
Enso is a _polyglot programming_ languge.
1413

15-
It is broken down into the following sections:
14+
### interoperability with Java
15+
16+
Unlike many other programming languages the _system language_ of Enso (e.g. the
17+
language that is used to do low-level operating system tasks) is **Java**. As
18+
such a lot of attention has been dedicated to make interoperability with
19+
**Java** as smooth as possible:
20+
21+
- [**Java:**](./java.md) Detailed info about the Java polyglot bindings.
22+
23+
Many `Standard` libraries are using these `polyglot java import` statements.
24+
Custom projects and libraries are encouraged to do the same. Interoperability
25+
with Java is a first class citizen in the Enso programming language.
26+
27+
### Interoperability with Python, JavaScript & co.
28+
29+
Enso greatly benefits from the
30+
[polyglot ecosystem of GraalVM](http://graalvm.org) and easily interops with any
31+
language from that ecosystem. Including **JavaScript**, **Python**, **R**, etc.
32+
33+
- [**Python:**](./python.md) Specifics of the Python polyglot bindings.
34+
35+
Interop with these _dynamic languages_ is primarily supported via
36+
[foreign function definitions](./polyglot-bindings.md#foreign-functions). When
37+
accessing Python, as well as other dynamic languages, the same pattern is used.
38+
Enough include the language support in the
39+
[distribution](../distribution/distribution.md) and the language gets
40+
automatically exposed via `foreign` directive to Enso programs.
41+
42+
## Implementation
43+
44+
Additional overview is provided in following documents:
1645

1746
- [**Polyglot Bindings:**](./polyglot-bindings.md) A document providing an
1847
overview of the mechanisms provided to work with polyglot bindings in Enso.
1948
- [**Typing Polyglot Bindings:**](./typing-polyglot-bindings.md) An exploration
2049
of how we can provide a modicum of type safety for the polyglot bindings in
2150
Enso.
2251

23-
It also provides language-specific documentation for the various supported
24-
polyglot languages. These are as follows:
25-
26-
- [**Java:**](./java.md) Information specific to the Java polyglot bindings.
27-
- [**Python:**](./python.md) Information specific to the Python polyglot
28-
bindings.
52+
Implementation details are described in
53+
[EpbLanguage javadoc](../../engine/runtime-language-epb/src/main/java/org/enso/interpreter/epb/EpbLanguage.java).

docs/polyglot/polyglot-bindings.md

Lines changed: 73 additions & 133 deletions
Original file line numberDiff line numberDiff line change
@@ -8,98 +8,19 @@ order: 1
88

99
# Polyglot Bindings
1010

11-
This document deals with the specification and design for the polyglot interop
12-
system provided in the Enso runtime. This system allows users to connect Enso to
13-
other supported programming languages, to both provide access to a wealth of
14-
libraries, and to integrate Enso into existing systems.
15-
16-
The polyglot support in Enso is best-in class, and it supports this through two
17-
main mechanisms:
18-
19-
1. **Polyglot FFI:** The low-level polyglot support provides a fairly low-level
20-
syntax sugar system for working with values from foreign languages.
21-
2. **Embedded Syntax:** This system allows users to write code from other
22-
languages directly in their `.enso` files, and to seamlessly share values
23-
between Enso and that foreign code.
24-
25-
<!-- MarkdownTOC levels="2,3" autolink="true" -->
26-
27-
- [Impedance Mismatch](#impedance-mismatch)
28-
- [The Polyglot FFI](#the-polyglot-ffi)
29-
- [Importing Polyglot Bindings](#importing-polyglot-bindings)
30-
- [Using Polyglot Bindings](#using-polyglot-bindings)
31-
- [Importing Polyglot Bindings \(Syntax\)](#importing-polyglot-bindings-syntax)
32-
- [Using Polyglot Bindings \(Syntax\)](#using-polyglot-bindings-syntax)
33-
- [Finding Polyglot Bindings](#finding-polyglot-bindings)
34-
- [Embedded Syntax](#embedded-syntax)
35-
- [Embedded Syntax Usage \(Syntax\)](#embedded-syntax-usage-syntax)
36-
37-
<!-- /MarkdownTOC -->
11+
Enso provides [robust interoperability](./README.md) with other programming
12+
languages. This document describes how users can connect Enso to other supported
13+
programming languages to gain access to a wealth of libraries, as well as to
14+
integrate Enso into existing systems.
3815

39-
## Impedance Mismatch
40-
41-
Polyglot interoperation in Enso has a significant impedance mismatch. In
42-
essence, this means that there is a mismatch between Enso's language semantics
43-
and the semantics of the foreign languages that are being worked with.
44-
45-
While some of this mismatch can be worked around by manually wrapping the
46-
foreign constructs in Enso, there are still concepts that can't easily be
47-
represented by Enso.
48-
49-
> The actionables for this section are:
50-
>
51-
> - Expand on the impedance mismatch and how it leads to the defined semantics.
52-
53-
## The Polyglot FFI
54-
55-
The low-level polyglot FFI mechanism refers to a way to use polyglot objects
56-
directly in Enso code. This can be used to underlie a library implementaion in
57-
Enso, or to interoperate with code running in other languages.
58-
59-
The mechanism provides users with the facilities to import bindings from other
60-
languages and call them via a generic mechanism.
61-
62-
### Importing Polyglot Bindings
63-
64-
When importing a polyglot binding into scope in an Enso file, this introduces a
65-
_polyglot object_ into scope. This object will have appropriate fields and/or
66-
methods defined on it, as described by the foreign language implementation.
67-
68-
> The actionables for this section are:
69-
>
70-
> - Expand greatly on the detail of this as the semantics of the imports become
71-
> clearer.
72-
73-
### Using Polyglot Bindings
74-
75-
With a polyglot object in scope, the user is free to call methods on it
76-
directly. These polyglot objects are inherently dynamically typed, meaning that
77-
any operation may _fail_ at runtime.
78-
79-
Enso implements a generic variadic syntax for calling polyglot functions using
80-
vectors of arguments. In essence, this is necessary due to the significant
81-
impedance mismatch between Enso's runtime semantics (let alone the type system)
82-
and the runtime semantics of many of the polyglot languages.
83-
84-
We went the way of the variadic call for multiple reasons:
85-
86-
- It allows us to match up with a wide range of language semantics (such as
87-
subtyping and overloading).
88-
- It is flexible and easy to expand in the future.
89-
- We can easily build a more Enso-feeling interface on top of it.
90-
91-
By way of illustrative example, Java supports method overloading and subtyping,
92-
two things which have no real equivalent in the Enso type system.
16+
## `polyglot import`
9317

94-
> The actionables for this section are:
95-
>
96-
> - Expand greatly on the runtime semantics of working with polyglot bindings.
97-
> - Determine how to make the inherent 'failability' of polyglot objects safer.
18+
Accessing existing objects of foreign languages can be done via
19+
`polyglot xyz import` statements. This primarily works for **Java** classes:
9820

99-
### Importing Polyglot Bindings (Syntax)
21+
- [**Java:**](./java.md) Detailed info about the Java polyglot bindings.
10022

101-
Polyglot bindings can be imported using a polyglot import directive. This is
102-
constructed as follows:
23+
The _polyglot import directive_ is constructed as follows:
10324

10425
- The `polyglot` keyword
10526
- A language identifier (e.g. `java`).
@@ -113,39 +34,41 @@ For example:
11334

11435
```ruby
11536
polyglot java import org.example.MyClass as MyClassJava
116-
polyglot c import struct NetworkPacket as NetworkPacketC
37+
polyglot c import struct NetworkPacket
11738
```
11839

119-
### Using Polyglot Bindings (Syntax)
120-
121-
A polyglot binding is a polyglot object that has methods and/or fields defined
122-
on it. Due to an impedance mismatch between languages, Enso implements a
123-
variadic syntax for calling these polyglot bindings using vectors.
124-
125-
In essence, we have a primitive function as follows:
40+
Once imported the `MyClassJava` as well as `NetworkPacket` objects behave as
41+
`Any` Enso objects. Such objects have methods and/or fields defined on them. The
42+
following is a valid usage of a polyglot binding:
12643

12744
```ruby
128-
Polyglot.method : Polyglot.Object -> [Any] -> Any
45+
main =
46+
x = MyClassJava.foo [1, 2, 3] # a static method
47+
inst = MyClassJava.new [a, b, c] # a constructor
48+
bar = inst.method [x, y] # an instance method
12949
```
13050

131-
It works as follows:
51+
### Using Polyglot Bindings
13252

133-
- It is a method called `method` defined on the `Polyglot` type. The name
134-
`method` is, however, a stand-in for the name of the method in question.
135-
- It takes an object instance of the polyglot object.
136-
- It takes a vector of arguments (and is hence variadic).
137-
- And it returns some value.
53+
With a polyglot object in scope, the user is free to call methods on it
54+
directly. These polyglot objects are inherently dynamically typed, meaning that
55+
they have `Any` type. As such there is no _static type checking_ when invoking
56+
methods on such types and potential errors are only detected during runtime and
57+
result in a runtime _failure_ (a typical behavior of Python or JavaScript
58+
programs).
13859

139-
By way of example, the following is a valid usage of a polyglot binding:
60+
Enso implements a generic variadic syntax for calling polyglot functions using
61+
vectors of arguments. In essence, this is necessary due to the significant
62+
impedance mismatch between Enso's runtime semantics and the runtime semantics of
63+
many of the polyglot languages. Such a solution:
14064

141-
```ruby
142-
polyglot java import com.example.MyClass as MyClassJava
65+
- allows Enso to match up with a wide range of language semantics
66+
- for example Java's subtyping and overloading
67+
- it is flexible and easy to expand in the future.
68+
- allows building a more Enso-feeling interface on top of it.
14369

144-
main =
145-
x = MyClassJava.foo [1, 2, 3] # a static method
146-
inst = MyClassJava.new [a, b, c] # a constructor
147-
bar = inst.metod [x, y] # a static method
148-
```
70+
Thanks to the generic variadic syntax, it is possible to smoothly invoke Java
71+
overloaded and overriden methods.
14972

15073
### Finding Polyglot Bindings
15174

@@ -158,32 +81,49 @@ Inside each directory is an implementation-defined structure, with the polyglot
15881
implementation for that particular language needing to specify it. Please see
15982
the language-specific documentation for details.
16083

161-
## Embedded Syntax
84+
## `foreign` functions
85+
86+
It is possible to define new code snippets of foreign languages directly in
87+
`.enso` source files using _"Embedded Syntax"_. Such a handy support provides a
88+
truly smooth user experience:
16289

163-
The term "Embedded Syntax" is our terminology for the ability to use foreign
164-
language syntaxes directly from inside `.enso` files. This system builds upon
165-
the more generic mechanisms used by the [polyglot FFI](#the-polyglot-ffi) to
166-
provide a truly seamless user experience.
90+
```ruby
91+
foreign python concat x y = """
92+
def join(a, b):
93+
return str(a) + str(b)
94+
return join(x, y)
16795
168-
### Embedded Syntax Usage (Syntax)
96+
main u="Hello" s=" " w="World!" =
97+
concat (concat u s) w
98+
```
16999
170-
A polyglot block is introduced as follows:
100+
The previous example defines an Enso function `concat` that takes two arguments
101+
`a` and `b`. The function is implemented in Python. The Python code defines a
102+
local function `join` and uses it to compute and return result of `concat`. Then
103+
the `concat` function is invoked from a `main` Enso function to concatenate
104+
typical _Hello World!_ message.
171105
172-
- The `foreign` keyword starts a block.
173-
- This must be followed by a language identifier (e.g. `python`).
174-
- After the language identifier, the remaining syntax behaves like it is an Enso
175-
function definition until the `=`.
176-
- After the `=`, the user may write their foreign code as a string.
106+
- [**Python:**](./python.md) Details on Python polyglot bindings.
177107
178-
```ruby
179-
foreign python concat a b = """
180-
def concat(a, b):
181-
str(a) + str(b)
182-
```
108+
Similar syntax can be used for `js` and other dynamic languages. Certain
109+
languages require/have special support, but in general this mechanism is reusing
110+
polyglot capabilities of GraalVM Truffle framework and works with any language
111+
that implements its `InteropLibrary` and _"parse in a context"_ protocols.
183112
184-
In the above example, this defines a function `concat` that takes two arguments
185-
`a` and `b`, implemented in Python.
113+
## Impedance Mismatch
186114
187-
> The actionables for this section are:
188-
>
189-
> - Greatly flesh out the syntax for the high-level polyglot functionality.
115+
Enso is designed as a functional programming language and as such it assumes
116+
_mininal side effects_ when performing operation. Especially the _live
117+
programming_ environment provided by the Enso visual editor relies on operations
118+
being idempotent and having no side effects. Enso semantic enforces such _no
119+
side effects_ behavior for programs written in Enso.
120+
121+
This is not a typical behavior of other programming languages and certainly it
122+
is not enforced in languages like JavaScript, Python or Java. Polyglot
123+
interoperation in Enso has a significant impedance mismatch. In essence, this
124+
means that there is a mismatch between Enso's language semantics and the
125+
semantics of the foreign languages that are being worked with.
126+
127+
Some of thes mismatches can be worked around by manually wrapping the foreign
128+
constructs in Enso, however some just cannot. Care must be taken when dealing
129+
with other languages and especially their side-effects.

0 commit comments

Comments
 (0)