Function overload resolution

Summary

The current algorithm for function overload resolution is designed for greedy type inference. This RFC updates it for local type inference.

Motivation

The algorithm for function overload resolution determines which type to give f(x) for overloaded functions f. Currently it is as follows (simplified, for example specializing to one argument and result):

Let F by the type of f, and X be the type of x.
First flatten F to the form F = F1 &...& FN.
For each Fi
If Fi is a function type T -> U, try unifying X with T. If unification succeeds, return U.
If Fi is a free type, unify it with X -> Y (for fresh free type Y) and return Y.
Otherwise, report an error.
If we got here, overload resolution failed, so we bail and return the return type of one of the overloads.

For example:

if F is ((number? -> string?) & (string? -> number?) and X is free, then X is unified with number? and the result type is string?
if F is ((number? -> string?) & (string? -> number?) and X is (number | string), then an error is reported and the result type is string?
if F is ((number? -> string?) & (string? -> number?) and X is nil, then the result type is string?

The order of constraints for this algorithm matters, for example:

  type T = { f : (number -> string) & (string -> number) }
  local function useT(x : T) x.f("hi") end
  local function (x)
    x.f(0)  -- this unifies the type of x.f with number -> Y
    useT(x) -- this does not typecheck because T gives a more precise type for f
  end

but swapping the order of constraints:

  type T = { f : (number -> string) & (string -> number) }
  local function useT(x : T) x.f("hi") end
  local function (x)
    useT(x) -- unifies the type of x with T
    x.f(0)  -- this is fine
  end

This is okay for greedy type inference, where the order of constraints is important, but not great for local type inference, which is meant to be independent of constraint order.

Design

In this proposal, we delay producing a type for function applications until the function and argument types are concrete.

We do this by introducing types src F and tgt F(X) (syntax to be bikeshedded, but hopefully it will never be seen by creators!), standing for the parameter and result types of applying a function of type F to an argument of type X.

The algorithm for function overload resolution is now trivial: the type of f(x) is just tgt F(X), and this introduces a constraint X <: src(F).

The extra work is now in type normalization, which removes uses of src and tgt during quantification:

  src (T -> U) is  T 
  src (T & U)  is  src T | src U
  src (T | U)  is  src T & src U
  src T        is  src F when T is a table with a metatable with a __call__ property of type F
  src T        is  none in any other case

  tgt (T -> U) (V) is  U    when (T&V) is inhabited
  tgt (T & U) (V)  is  tgt T(V) & tgt U(V)
  tgt (T | U) (V)  is  tgt T(V) | tgt U(V)
  tgt T (V)        is  tgt F(V) when T is a table with a metatable with a __call__ property of type F
  tgt top (T)      is  top
  tgt T (V)        is  none in any other case

For example:

  src((number? -> string?) & (string? -> number?))                    is  (number | string)?
  tgt((number? -> string?) & (string? -> number?)) (number)           is  string?
  tgt((number? -> string?) & (string? -> number?)) (number | string)  is  (number | string)?
  tgt((number? -> string?) & (string? -> number?)) (nil)              is  nil

(This is a variant of the algorithm used in the Luau prototype.)

The real thing would need scaled up to type packs and generic functions.

Drawbacks

Anything we do about this will be a breaking change.

Scaling this up to type packs probably involves intersection and union on packs.

There may be other devils in the details.

We need to ensure src and tgt do not leak.

We need to make sure that normalization doesn't lose valuable autocomplete information.

We need an algorithm for "is a type inhabited".

Alternatives

We could try applying fixes to the current algorithm, and live with constraint order mattering.

We could adopt the Flow solution, which adds "if this then that" constraints, but that makes it very likely we will need to introduce additional backtracking.

4.3 KiB Raw Blame History