Native Interface

Really interlanguage interface.

Interface client's point of view

An import statement includes the optional foreign language name, which may be either a simple identifier or a string literal. For example

Loading...

Within a component, a foreign language import references a virtual API and component, both generated according to sensible rules for interoperation between Fortress and the other language. It may make sense to generate an actual API for reference purposes, and to avoid the burden of adding knowledge of the interlanguage interface to other tools.

Foreign-language on-demand imports ("import java java.util.String.{...} ") will not be supported for now.

Programmer-does-the-work IDL-like approaches are to be avoided.

Language processing point of view

From a language processing point of view, an API needs to be generated so that Fortress programs importing it make sense. The typechecker, interpreter, compiler, etc, are all written to expect that "import foo" actually finds an api named "foo".

One issue is how much of a type is imported. One endpoint is a simple opaque import; an other-language type is wrapped, has no methods, and the only operation available is unwrapping when passed to a native method. The other endpoint is full transitive generation of apis and interface glue for every type mentioned in the foreign import (for example, java String then implies StringBuffer, StringBuilder, Locale, CharSequence, and every type mentioned in the public members of those types, etc.) That is too much; it is more than Java does. Our goal is to wrap foreign objects and wrap their methods, and any types that they mention in their signatures will be imported opaquely. In the short team, we will implement opaque wrapping, which can then be used as the base/leaf case for the more involved type mapping.

From a Fortress point of view, wrapped foreign objects are value objects. That means that strict equality is NOT address equality.

Interpreter/compiler point of view

This is a moving target. We hope to be rid of the interpreter in a year, but till then, it is useful for continuity, testing, and validation. To interoperate with the interpreter, the gaskets generated for native code must implement various Java classes/interfaces internal to the interpreter. In the short run, the most important of these are Closure, Method, FObject, and FType. These get cleaned up as needed to make interoperation actually possible (such a refactoring can be seen in Environment, where an interface was created, and then generic behavior was factored into BaseEnv, which generated top level environments extend). This does not preclude generation of specialized interfaces for use between two pieces of compiled code; that is in fact the eventual goal of compiling Fortress.

It's important to make a distinction between type mapping, value mapping, and type relationship mapping. For example, it might be possible to map each foreign type one-for-one to a Fortress equivalent, but have no subtyping relations established between the Fortress equivalents. It's likely that values of a Fortress type might map to values of different types; for example ZZ32 can map to either "int" or "java.lang.Integer", as appropriate (we might want to formalize this with coercion).

Languages with privacy modifiers on types present other interesting issues. A Java method declared to return an interface type Bar must necessarily return a value that is an instance of some class, say Foo. Foo might not be a public type, and could easily be (and probably should be) inaccessible to the foreign function interface.

Java-specific issues and plans

This will probably grow its own page

import naming rules

import java java.foo.bar will wrap java PACKAGE java.foo.bar.

Wrapping (into a virtual/generated api) occurs at a package granularity. This is large, but if the granularity were at the class level, the name of the class ends up repeated in order to reference. For anti-example:

Loading...

then fully qualified references to the type would look like java.util.Date.Date. Yuck.

Package-as-api creates problems for naming static methods, which we expect map to top-level functions in the wrapper api. The naming convention is that static method f from some.java.pkg.Clss is imported as name Clss.f from api some.java.pkg. It seems, however, that there was no graceful way of escaping the need for foreign names with dots in them; static classes (Map.Entry, for example) also need to be named, and they can also contain their own static methods. This naming scheme also somewhat reduces the potential for clash between unrelated static method names, which means that unintentional (and likely invalid) overloadings will be less common than otherwise.

import implementation

There's two halves to the native import implementation. One half is generating a bogus API, adequate to fake out static analysis, compilation, and other tools. The other half is to generate the appropriate wrappers.

Because (we assume) that it is too expensive to wrap an entire package, we plan to do it in a demand-driven fashion. Provided that we never cache the fake API to disk, it can be custom generated for each compilation without the possibility of interference. Alternately, we can hash the contents and use that as the "name" of the cached data (we may need to do this because static analysis expects to write APIs to disk, and this will simplify the up-to-date story).

Suppose a repository (that is, the software implementing the repository foo) sees an import of a native API foo, and the import references member bar. Native APIs don't import any other Fortress APIs, but they may reference other Java types. So, for each Java package, the repository needs to maintain a record of what parts (what classes, mostly) have been imported from it, and for each class, whether it is imported "transparently" or "opaque". Any types implied by an import (e.g., import of a static function with parameter and return types) are recursively imported using this mechanism and added to a depends list for the package. This is not necessarily "the" depends list used by the repository. Once the closure (over Fortress imports) is finished, we know that there will be no more native imports, and we can render an API for type-checking purposes. At this point, the previously mentioned depends list is converted to import statements in the generated API.

When generating code, it probably works better to change the organization. Instead of generating one class for an entire "component", here it makes more sense to work class (trait/object) by class, and to embed code for static methods and top-level variables into that class. The name matching (that would be tedious/annoying to a human programmer) is mechanical, and this avoids the problem of caching different code for the API based on what imports are made from it.

primitive types and values

Java Object, String, char, boolean, byte, short, int, long, float, double get special treatment (special mapping) to and from the obviously corresponding Fortress types. The same goes for the boxed variants of the unboxed primitive types.

type relationships

Java interfaces map one-for-one with Fortress traits (what about non-public interfaces? How much can we access reflectively, are we forced to parse the Java?). If Java interface I extends J,K,L, the Fortress trait T(I) extends T(J),T(K),T(L) .

If everything were public, a non-final Java class C corresponds to a Fortress trait T(C) and a Fortress object O(C), where O(C) extends T(C). Because Fortress objects cannot be extended, if another Java class D extends C, then the relationship is modeled by T(D) extends T(C) and O(D) extends T(D). The trait types are what gets used in all signatures, but the object types are the actual Fortress type of the wrapper.

implementation plan

• opaque import (no methods, no type relationships)
  Need to resolve import and name of (e.g.) java.util.Date.
  Example here.
• primitives and static methods
  
• constructors (special case of static method)
• arrays. Arrays seem like a special case; an array from Java can be wrapped into a Fortress array pretty easily. Converting a Fortress array into a Java array will usually require allocation and copying. However, some of the new Java libraries use (instead of arrays) interfaces like CharSequence or Iterable.
• methods
• fields
• type relationships

Somewhere in here we need to think about overloading.

Overloading issues

Overloading depends on type relationships. There may be too many problems. Some things are known:

• Java final class is equivalent to Fortress object.
• Blind translation of Java methods into Fortress methods will lead to violations of Fortress overloading rules; Java performs ambiguity checks at use, not declaration; Fortress enforces unambiguous sets of declarations. Note that this is only a problem if Java subtyping relationships are translated into (the same) Fortress subtyping relationships. This seems to force a tradeoff between automatic import of all visible static functions, or automatic import of subtype relationships (if either one is relaxed, then it's ok, maybe).
• Two Java classes are either in a subtype relationship or in an exclusion relationship. The number of exclusion relationships is larger than we necessarily want to enumerate; perhaps the wrapping process will only generate the exclusions needed to disambiguate method calls that are visible.
• Subtype queries are available (and relatively cheap) even when the Java class in question cannot be named because it is non-public.

Here are some proposed attacks on the overloading problem:

• Use special dispatch rules for methods known to be Java methods (will this always be statically visible)?
• Instead, do an AST-level translation, so that f(x,y) is rewritten to f(x ASIF S, y ASIF T) where S and T are the static types of f's parameters.
• Automatically generate the completion (the meet function) with an exception-throwing body (is this always possible? Is the meet sayable?)
• BAN IT. Put the problem in the importer's lap, with the workaround
  Loading...
  That is, rename Cls.f as f, where its overloading can be modified by an additional definition of f.

Notice that there's some lurking problems here; potentially unsayable meets, and interesting assumptions about what will or will not be statically observable.

Fortran-specific issues and plans

We should review the work done in the  Babel system.