Added first draft of rfcs/recursive-type-unrestriction.md

rfcs/recursive-type-unrestriction.md

@@ -0,0 +1,121 @@
+# Loosening the recursive type restriction
+
+## Summary
+
+Luau supports recursive type aliases, but with an important
+restriction: users can declare functions of recursive types, but *not*
+recursive type functions. This has problems with sophisticated uses of
+types, for example ones which mix recursive types and nested generics.
+This RFC proposes loosening this restriction.
+
+## Motivation
+
+Luau supports recursive type aliases, but with an important restriction:
+users can declare functions of recursive types, such as:
+```lua
+  type Tree<a> = { data: a, children: {Tree<a>} }
+```
+but *not* recursive type functions, such as:
+```lua
+  type TreeWithMap<a> = { ..., map: <b>(a -> b) -> Tree<b> }
+
+```
+These examples cope up naturally in OO code bases with generic types.
+
+## Design
+
+*This section to be filled in once we decide which alternative to use*
+
+## Drawbacks
+
+*This section to be filled in once we decide which alternative to use*
+
+## Alternatives
+
+### Lazy recursive type instantiations
+
+The most permissive change would be to make recursive type
+instantiation lazy rather than strict. In this approach `T<U>` would
+not be instantiated immediately, but only when the body is needed. In
+particular, during unification we can unify `T<U>` with `T<V>` by
+first trying to unify `U` and `V`, and only if that fails try to unify
+the instantiations.
+
+*Advantages*: this allows recursive types with infinite expansions like:
+```lua
+  type Foo<T> = { ..., promises: {Foo<Promise<T>>} }
+```
+
+### Lazy recursive type instantiations with a cache
+
+As above, but keep a cache for each type function.
+
+*Advantages*: reduces the size of the type graph.
+
+### Strict recursive type instantiations with a cache
+
+Rather than lazily instantiating type functions when they are used, we
+could carry on instantiating them when they are defined, and use a
+cache to reuse them. In particular, the cache would be populated when the
+recursive types are defined, and used when types are used recursively.
+
+For example:
+```
+type T<a,b> = { foo: T<b,number>? } 
+```
+would result in cache entries:
+```
+T<a,b> = { foo: T<b,number>? }
+T<b,number> = { foo: T<number,number>? }
+T<number,number> = { foo: T<number,number>? }
+```
+This can result in exponential blowup, for example:
+```
+type T<a,b> = { foo: T<b,number>?, bar: T<b,string>? } 
+```
+would result in cache entries:
+```
+T<a,b> = { foo: T<b,number>?, bar: T<b,string>? }
+T<b,number> = { foo: T<number,number>?, bar: T<string,number>? }
+T<b,string> = { foo: T<string,number>?, bar: T<string,string>? }
+T<number,number> = { foo: T<number,number>?, bar: T<number,string>? }
+T<number,string> = { foo: T<string,number>?, bar: T<string,string>? }
+T<string,number> = { foo: T<number,number>?, bar: T<number,string>? }
+T<string,string> = { foo: T<string,number>?, bar: T<string,string>? }
+```
+Applying this to a type function with N type variables results in more than 2^N
+types. Because of blowup, we would need a bound on cache size.
+
+This can also result in the cache being exhausted, for example:
+```
+type T<a> = { foo: T<Promise<a>>? } 
+```
+results in an infinite type graph with cache:
+```
+T<a> = { foo: T<Promise<a>>? } 
+T<Promise<a>> = { foo: T<Promise<Promise<a>>>? } 
+T<Promise<Promise<a>>> = { foo: T<Promise<Promise<Promise<a>>>>? }
+...
+```
+
+*Advantages*: types are computed strictly, so we don't have to worry about lazy types
+producing unbounded type graphs during unification.
+
+### Strict recursive type instantiations with a cache and an occurrence check
+
+We can use occurrence checks to ensure there's no blowup. We can restrict
+a recursive use `T<U1,...,UN>` in the definition of `T<a1...aN>` so that either `UI` is `aI`
+or contains none of `a1...aN`. For example this bans
+```
+type T<a> = { foo: T<Promise<a>>? } 
+```
+since `Promise<a>` is not `a` but contains `a`, and bans
+```
+type T<a,b> = { foo: T<b,number>? } 
+```
+since `a` is not `b`, but allows:
+```
+type T<a,b> = { foo: T<a,number>? } 
+```
+
+*Advantages*: types are computed strictly, and may produce better error messages if the occurs check fails.