Merge branch 'master' into merge

docs/_pages/typecheck.md

@@ -31,20 +31,6 @@ foo = 1
 
 However, given the second snippet in strict mode, the type checker would be able to infer `number` for `foo`.
 
-## Unknown symbols
-
-Consider how often you're likely to assign a new value to a local variable. What if you accidentally misspelled it? Oops, it's now assigned globally and your local variable is still using the old value.
-
-```lua
-local someLocal = 1
-
-soeLocal = 2 -- the bug
-
-print(someLocal)
-```
-
-Because of this, Luau type checker currently emits an error in strict mode; use local variables instead.
-
 ## Structural type system
 
 Luau's type system is structural by default, which is to say that we inspect the shape of two tables to see if they are similar enough. This was the obvious choice because Lua 5.1 is inherently structural.
@@ -267,6 +253,23 @@ Note: it's impossible to create an intersection type of some primitive types, e.
 
 Note: Luau still does not support user-defined overloaded functions. Some of Roblox and Lua 5.1 functions have different function signature, so inherently requires overloaded functions.
 
+## Singleton types (aka literal types)
+
+Luau's type system also supports singleton types, which means it's a type that represents one single value at runtime. At this time, both string and booleans are representable in types.
+
+> We do not currently support numbers as types. For now, this is intentional.
+
+```lua
+local foo: "Foo" = "Foo" -- ok
+local bar: "Bar" = foo   -- not ok
+local baz: string = foo  -- ok
+
+local t: true = true -- ok
+local f: false = false -- ok
+```
+
+This happens all the time, especially through [type refinements](#type-refinements) and is also incredibly useful when you want to enforce program invariants in the type system! See [tagged unions](#tagged-unions) for more information.
+
 ## Variadic types
 
 Luau permits assigning a type to the `...` variadic symbol like any other parameter:
@@ -375,22 +378,40 @@ local account: Account = Account.new("Alexander", 500)
              --^^^^^^^ not ok, 'Account' does not exist
 ```
 
+## Tagged unions
+
+Tagged unions are just union types! In particular, they're union types of tables where they have at least _some_ common properties but the structure of the tables are different enough. Here's one example:
+
+```lua
+type Result<T, E> = { type: "ok", value: T } | { type: "err", error: E }
+```
+
+This `Result<T, E>` type can be discriminated by using type refinements on the property `type`, like so:
+
+```lua
+if result.type == "ok" then
+    -- result is known to be { type: "ok", value: T }
+    -- and attempting to index for error here will fail
+    print(result.value)
+elseif result.type == "err" then
+    -- result is known to be { type: "err", error: E }
+    -- and attempting to index for value here will fail
+    print(result.error)
+end
+```
+
+Which works out because `value: T` exists only when `type` is in actual fact `"ok"`, and `error: E` exists only when `type` is in actual fact `"err"`.
+
 ## Type refinements
 
-When we check the type of a value, what we're doing is we're refining the type, hence "type refinement." Currently, the support for this is somewhat basic.
+When we check the type of any lvalue (a global, a local, or a property), what we're doing is we're refining the type, hence "type refinement." The support for this is arbitrarily complex, so go crazy!
 
-Using `type` comparison:
-```lua
-local stringOrNumber: string | number = "foo"
+Here are all the ways you can refine:
+1. Truthy test: `if x then` will refine `x` to be truthy.
+2. Type guards: `if type(x) == "number" then` will refine `x` to be `number`.
+3. Equality: `x == "hello"` will refine `x` to be a singleton type `"hello"`.
 
-if type(x) == "string" then
-    local onlyString: string = stringOrNumber -- ok
-    local onlyNumber: number = stringOrNumber -- not ok
-end
-
-local onlyString: string = stringOrNumber -- not ok
-local onlyNumber: number = stringOrNumber -- not ok
-```
+And they can be composed with many of `and`/`or`/`not`. `not`, just like `~=`, will flip the resulting refinements, that is `not x` will refine `x` to be falsy.
 
 Using truthy test:
 ```lua
@@ -398,10 +419,55 @@ local maybeString: string? = nil
 
 if maybeString then
     local onlyString: string = maybeString -- ok
+    local onlyNil: nil = maybeString       -- not ok
+end
+
+if not maybeString then
+    local onlyString: string = maybeString -- not ok
+    local onlyNil: nil = maybeString       -- ok
 end
 ```
 
-And using `assert` will work with the above type guards:
+Using `type` test:
+```lua
+local stringOrNumber: string | number = "foo"
+
+if type(stringOrNumber) == "string" then
+    local onlyString: string = stringOrNumber -- ok
+    local onlyNumber: number = stringOrNumber -- not ok
+end
+
+if type(stringOrNumber) ~= "string" then
+    local onlyString: string = stringOrNumber -- not ok
+    local onlyNumber: number = stringOrNumber -- ok
+end
+```
+
+Using equality test:
+```lua
+local myString: string = f()
+
+if myString == "hello" then
+    local hello: "hello" = myString -- ok because it is absolutely "hello"!
+    local copy: string = myString   -- ok
+end
+```
+
+And as said earlier, we can compose as many of `and`/`or`/`not` as we wish with these refinements:
+```lua
+local function f(x: any, y: any)
+    if (x == "hello" or x == "bye") and type(y) == "string" then
+        -- x is of type "hello" | "bye"
+        -- y is of type string
+    end
+
+    if not (x ~= "hi") then
+        -- x is of type "hi"
+    end
+end
+```
+
+`assert` can also be used to refine in all the same ways:
 ```lua
 local stringOrNumber: string | number = "foo"
 
@@ -411,7 +477,7 @@ local onlyString: string = stringOrNumber -- ok
 local onlyNumber: number = stringOrNumber -- not ok
 ```
 
-## Typecasts
+## Type casts
 
 Expressions may be typecast using `::`.  Typecasting is useful for specifying the type of an expression when the automatically inferred type is too generic.
 
@@ -487,4 +553,4 @@ There are some caveats here though. For instance, the require path must be resol
 Cyclic module dependencies can cause problems for the type checker.  In order to break a module dependency cycle a typecast of the module to `any` may be used:
 ```lua
 local myModule = require(MyModule) :: any
-```
+```

docs/_posts/2022-03-31-luau-recap-march-2022.md

@@ -0,0 +1,109 @@
+---
+layout: single
+title:  "Luau Recap: March 2022"
+---
+
+Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org).
+
+[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-march-2022/).]
+
+## Singleton types
+
+We added support for singleton types! These allow you to use string or
+boolean literals in types. These types are only inhabited by the
+literal, for example if a variable `x` has type `"foo"`, then `x ==
+"foo"` is guaranteed to be true.
+
+Singleton types are particularly useful when combined with union types,
+for example:
+
+```lua
+type Animals = "Dog" | "Cat" | "Bird"
+```
+
+or:
+
+```lua
+type Falsey = false | nil
+```
+
+In particular, singleton types play well with unions of tables,
+allowing tagged unions (also known as discriminated unions):
+
+```lua
+type Ok<T> = { type: "ok", value: T }
+type Err<E> = { type: "error", error: E }
+type Result<T, E> = Ok<T> | Err<E>
+
+local result: Result<number, string> = ...
+if result.type == "ok" then
+    -- result :: Ok<number>
+    print(result.value)
+elseif result.type == "error" then
+    -- result :: Err<string>
+    error(result.error)
+end
+```
+
+The RFC for singleton types is https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md
+
+## Width subtyping
+
+A common idiom for programming with tables is to provide a public interface type, but to keep some of the concrete implementation private, for example:
+
+```lua
+type Interface = {
+    name: string,
+}
+
+type Concrete = {
+    name: string,
+    id: number,
+}
+```
+
+Within a module, a developer might use the concrete type, but export functions using the interface type:
+
+```lua
+local x: Concrete = {
+    name = "foo",
+    id = 123,
+}
+
+local function get(): Interface
+    return x
+end
+```
+
+Previously examples like this did not typecheck but now they do!
+
+This language feature is called *width subtyping* (it allows tables to get *wider*, that is to have more properties).
+
+The RFC for width subtyping is https://github.com/Roblox/luau/blob/master/rfcs/sealed-table-subtyping.md
+
+## Typechecking improvements
+
+ * Generic function type inference now works the same for generic types and generic type packs.
+ * We improved some error messages.
+ * There are now fewer crashes (hopefully none!) due to mutating types inside the Luau typechecker.
+ * We fixed a bug that could cause two incompatible copies of the same class to be created.
+ * Luau now copes better with cyclic metatable types (it gives a type error rather than hanging).
+ * Fixed a case where types are not properly bound to all of the subtype when the subtype is a union.
+ * We fixed a bug that confused union and intersection types of table properties.
+ * Functions declared as `function f(x : any)` can now be called as `f()` without a type error.
+
+## API improvements
+
+ * Implement `table.clone` which takes a table and returns a new table that has the same keys/values/metatable. The cloning is shallow - if some keys refer to tables that need to be cloned, that can be done manually by modifying the resulting table.
+
+## Debugger improvements
+
+ * Use the property name as the name of methods in the debugger.
+
+## Performance improvements
+
+ * Optimize table rehashing (~15% faster dictionary table resize on average)
+ * Improve performance of freeing tables (~5% lift on some GC benchmarks)
+ * Improve gathering performance metrics for GC.
+ * Reduce stack memory reallocation.
+

rfcs/STATUS.md

@@ -17,17 +17,10 @@ This document tracks unimplemented RFCs.
 
 ## Sealed/unsealed typing changes
 
-[RFC: Unsealed table literals](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-literals.md) |
 [RFC: Only strip optional properties from unsealed tables during subtyping](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-subtyping-strips-optional-properties.md)
 
 **Status**: Implemented but not fully rolled out yet.
 
-## Singleton types
-
-[RFC: Singleton types](https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md)
-
-**Status**: Implemented but not fully rolled out yet.
-
 ## Safe navigation operator
 
 [RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md)
@@ -47,3 +40,9 @@ This document tracks unimplemented RFCs.
 [RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md)
 
 **Status**: Needs implementation
+
+## Lower Bounds Calculation
+
+[RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md)
+
+**Status**: Needs implementation

rfcs/function-bit32-countlz-countrz.md

@@ -1,11 +1,11 @@
 # bit32.countlz/countrz
 
+**Status**: Implemented
+
 ## Summary
 
 Add bit32.countlz (count left zeroes) and bit32.countrz (count right zeroes) to accelerate bit scanning
 
-**Status**: Implemented
-
 ## Motivation
 
 All CPUs have instructions to determine the position of first/last set bit in an integer. These instructions have a variety of uses, the popular ones being:

rfcs/function-coroutine-close.md

@@ -1,11 +1,11 @@
 # coroutine.close
 
+**Status**: Implemented
+
 ## Summary
 
 Add `coroutine.close` function from Lua 5.4 that takes a suspended coroutine and makes it "dead" (non-runnable).
 
-**Status**: Implemented
-
 ## Motivation
 
 When implementing various higher level objects on top of coroutines, such as promises, it can be useful to cancel the coroutine execution externally - when the caller is not

rfcs/lower-bounds-calculation.md

@@ -0,0 +1,217 @@
+# Lower Bounds Calculation
+
+## Summary
+
+We propose adapting lower bounds calculation from Pierce's Local Type Inference paper into the Luau type inference algorithm.
+
+https://www.cis.upenn.edu/~bcpierce/papers/lti-toplas.pdf
+
+## Motivation
+
+There are a number of important scenarios that occur where Luau cannot infer a sensible type without annotations.
+
+Many of these revolve around type variables that occur in contravariant positions.
+
+### Function Return Types
+
+A very common thing to write in Luau is a function to try to find something in some data structure.  These functions habitually return the relevant datum when it is successfully found, or `nil` in the case that it cannot.  For instance:
+
+```lua
+-- A.lua
+function find_first_if(vec, f)
+    for i, e in ipairs(vec) do
+        if f(e) then
+            return i
+        end
+    end
+
+    return nil
+end
+```
+
+This function has two `return` statements: One returns `number` and the other `nil`.  Today, Luau flags this as an error.  We ask authors to add a return annotation to make this error go away.
+
+We would like to automatically infer `find_first_if : <T>({T}, (T) -> boolean) -> number?`.
+
+Higher order functions also present a similar problem.
+
+```lua
+-- B.lua
+function foo(f)
+    f(5)
+    f("string")
+end
+```
+
+There is nothing wrong with the implementation of `foo` here, but Luau fails to typecheck it all the same because `f` is used in an inconsistent way.  This too can be worked around by introducing a type annotation for `f`.
+
+The fact that the return type of `f` is never used confounds things a little, but for now it would be a big improvement if we inferred `f : <T...>((number | string) -> T...) -> ()`.
+
+## Design
+
+We introduce a new kind of TypeVar, `ConstrainedTypeVar` to represent a TypeVar whose lower bounds are known.  We will never expose syntax for a user to write these types: They only temporarily exist as type inference is being performed.
+
+When unifying some type with a `ConstrainedTypeVar` we _broaden_ the set of constraints that can be placed upon it.
+
+It may help to realize that what we have been doing up until now has been _upper bounds calculation_.
+
+When we `quantify` a function, we will _normalize_ each type and convert each `ConstrainedTypeVar` into a `UnionTypeVar`.
+
+### Normalization
+
+When computing lower bounds, we need to have some process by which we reduce types down to a minimal shape and canonicalize them, if only to have a clean way to flush out degenerate unions like `A | A`.  Normalization is about reducing union and intersection types to a minimal, canonicalizable shape.
+
+A normalized union is one where there do not exist two branches on the union where one is a subtype of the other.  It is quite straightforward to implement.
+
+A normalized intersection is a little bit more complicated:
+
+1. The tables of an intersection are always combined into a single table.  Coincident properties are merged into intersections of their own.
+    * eg `normalize({x: number, y: string} & {y: number, z: number}) == {x: number, y: string & number, z: number}`
+    * This is recursive.  eg `normalize({x: {y: number}} & {x: {y: string}}) == {x: {y: number & string}}`
+1. If two functions in the intersection have a subtyping relationship, the normalization results only in the super-type-most function. (more on function subtyping later)
+
+### Function subtyping relationships
+
+If we are going to infer intersections of functions, then we need to be very careful about keeping combinatorics under control.  We therefore need to be very deliberate about what subtyping rules we have for functions of differing arity.  We have some important requirements:
+
+* We'd like some way to canonicalize intersections of functions, and yet
+* optional function arguments are a great feature that we don't want to break
+
+A very important use case for us is the case where the user is providing a callback to some higher-order function, and that function will be invoked with extra arguments that the original customer doesn't actually care about.  For example:
+
+```lua
+-- C.lua
+function map_array(arr, f)
+    local result = {}
+    for i, e in ipairs(arr) do
+        table.insert(result, f(e, i, arr))
+    end
+    return result
+end
+
+local example = {1, 2, 3, 4}
+local example_result = map_array(example, function(i) return i * 2 end)
+```
+
+This function mirrors the actual `Array.map` function in JavaScript.  It is very frequent for users of this function to provide a lambda that only accepts one argument.  It would be annoying for callers to be forced to provide a lambda that accepts two unused arguments.  This obviously becomes even worse if the function later changes to provide yet more optional information to the callback.
+
+This use case is very important for Roblox, as we have many APIs that accept callbacks.  Implementors of those callbacks frequently omit arguments that they don't care about.
+
+Here is an example straight out of the Roblox developer documentation. ([full example here](https://developer.roblox.com/en-us/api-reference/event/BasePart/Touched))
+
+```lua
+-- D.lua
+local part = script.Parent
+
+local function blink()
+    -- ...
+end
+
+part.Touched:Connect(blink)
+```
+
+The `Touched` event actually passes a single argument: the part that touched the `Instance` in question.  In this example, it is omitted from the callback handler.
+
+We therefore want _oversaturation_ of a function to be allowed, but this combines with optional function arguments to create a problem with soundness.  Consider the following:
+
+```lua
+-- E.lua
+type Callback = (Instance) -> ()
+
+local cb: Callback
+function register_callback(c: Callback)
+    cb = c
+end
+
+function invoke_callback(i: Instance)
+    cb(i)
+end
+
+---
+
+function bad_callback(x: number?)
+end
+
+local obscured: () -> () = bad_callback
+
+register_callback(obscured)
+
+function good_callback()
+end
+
+register_callback(good_callback)
+```
+
+The problem we run into is, if we allow the subtyping rule `(T?) -> () <: () -> ()` and also allow oversaturation of a function, it becomes easy to obscure an argument type and pass the wrong type of value to it.
+
+Next, consider the following type alias
+
+```lua
+-- F.lua
+type OldFunctionType = (any, any) -> any
+type NewFunctionType = (any) -> any
+type FunctionType = OldFunctionType & NewFunctionType
+```
+
+If we have a subtyping rule `(T0..TN) <: (T0..TN-1)` to permit the function subtyping relationship `(T0..TN-1) -> R <: (T0..TN) -> R`, then the above type alias normalizes to `(any) -> any`.  In order to call the two-argument variation, we would need to permit oversaturation, which runs afoul of the soundness hole from the previous example.
+
+We need a solution here.
+
+To resolve this, let's reframe things in simpler terms:
+
+If there is never a subtyping relationship between packs of different length, then we don't have any soundness issues, but we find ourselves unable to register `good_callback`.
+
+To resolve _that_, consider that we are in truth being a bit hasty when we say `good_callback : () -> ()`.  We can pass any number of arguments to this function safely.  We could choose to type `good_callback : () -> () & (any) -> () & (any, any) -> () & ...`.  Luau already has syntax for this particular sort of infinite intersection: `good_callback : (any...) -> ()`.
+
+So, we propose some different inference rules for functions:
+
+1. The AST fragment `function(arg0..argN) ... end` is typed `(T0..TN, any...) -> R` where `arg0..argN : T0..TN` and `R` is the inferred return type of the function body.  Function statements are inferred the same way.
+1. Type annotations are unchanged.  `() -> ()` is still a nullary function.
+
+For reference, the subtyping rules for unions and functions are unchanged.  We include them here for clarity.
+
+1. `A <: A | B`
+1. `B <: A | B`
+1. `A | B <: T` if `A <: T` or `B <: T`
+1. `T -> R <: U -> S` if `U <: T` and `R <: S`
+
+We propose new subtyping rules for type packs:
+
+1. `(T0..TN) <: (U0..UN)` if, for each `T` and `U`, `T <: U`
+1. `(U...)` is the same as `() | (U) | (U, U) | (U, U, U) | ...`, therefore
+1. `(T0..TN) <: (U...)` if for each `T`, `T <: U`, therefore
+1. `(U...) -> R <: (T0..TN) -> R` if for each `T`, `T <: U`
+
+The important difference is that we remove all subtyping rules that mention options.  Functions of different arities are no longer considered subtypes of one another.  Optional function arguments are still allowed, but function as a feature of function calls.
+
+Under these rules, functions of different arities can never be converted to one another, but actual functions are known to be safe to oversaturate with anything, and so gain a type that says so.
+
+Under these subtyping rules, snippets `C.lua` and `D.lua`, check the way we want: literal functions are implicitly safe to oversaturate, so it is fine to cast them as the necessary callback function type.
+
+`E.lua` also typechecks the way we need it to: `(Instance) -> () </: () -> ()` and so `obscured` cannot receive the value `bad_callback`, which prevents it from being passed to `register_callback`.  However, `good_callback : (any...) -> ()` and `(any...) -> () <: (Instance) -> ()` and so it is safe to register `good_callback`.
+
+Snippet `F.lua` is also fixed with this ruleset: There is no subtyping relationship between `(any) -> ()` and `(any, any) -> ()`, so the intersection is not combined under normalization.
+
+This works, but itself creates some small problems that we need to resolve:
+
+First, the `...` symbol still needs to be unavailable for functions that have been given this implicit `...any` type.  This is actually taken care of in the Luau parser, so no code change is required.
+
+Secondly, we do not want to silently allow oversaturation of direct calls to a function if we know that the arguments will be ignored.  We need to treat these variadic packs differently when unifying for function calls.
+
+Thirdly, we don't want to display this variadic in the signature if the author doesn't expect to see it.
+
+We solve these issues by adding a property `bool VariadicTypePack::hidden` to the implementation and switching on it in the above scenarios.  The implementation is relatively straightforward for all 3 cases.
+
+## Drawbacks
+
+There is a potential cause for concern that we will be inferring unions of functions in cases where we previously did not.  Unions are known to be potential sources of performance issues.  One possibility is to allow Luau to be less intelligent and have it "give up" and produce less precise types.  This would come at the cost of accuracy and soundness.
+
+If we allow functions to be oversaturated, we are going to miss out on opportunities to warn the user about legitimate problems with their program.  I think we will have to work out some kind of special logic to detect when we are oversaturating a function whose exact definition is known and warn on that.
+
+Allowing indirect function calls to be oversaturated with `nil` values only should be safe, but a little bit unfortunate.  As long as we statically know for certain that `nil` is actually a permissible value for that argument position, it should be safe.
+
+## Alternatives
+
+If we are willing to sacrifice soundness, we could adopt success typing and come up with an inference algorithm that produces less precise type information.
+
+We could also technically choose to do nothing, but this has some unpalatable consequences:  Something I would like to do in the near future is to have the inference algorithm assume the same `self` type for all methods of a table.  This will make inference of common OO patterns dramatically more intuitive and ergonomic, but inference of polymorphic methods requires some kind of lower bounds calculation to work correctly.

rfcs/syntax-safe-navigation-operator.md

@@ -1,5 +1,7 @@
 # Safe navigation postfix operator (?)
 
+**Note**: We have unresolved issues with interaction between this feature and Roblox instance hierarchy. This may affect the viability of this proposal.
+
 ## Summary
 
 Introduce syntax to navigate through `nil` values, or short-circuit with `nil` if it was encountered.

rfcs/syntax-singleton-types.md

@@ -2,6 +2,8 @@
 
 > Note: this RFC was adapted from an internal proposal that predates RFC process
 
+**Status**: Implemented
+
 ## Summary
 
 Introduce a new kind of type variable, called singleton types. They are just like normal types but has the capability to represent a constant runtime value as a type.

rfcs/syntax-type-ascription-bidi.md

@@ -1,11 +1,11 @@
 # Relaxing type assertions
 
+**Status**: Implemented
+
 ## Summary
 
 The way `::` works today is really strange.  The best solution we can come up with is to allow `::` to convert between any two related types.
 
-**Status**: Implemented
-
 ## Motivation
 
 Due to an accident of the implementation, the Luau `::` operator can only be used for downcasts and casts to `any`.

rfcs/unsealed-table-assign-optional-property.md

@@ -1,5 +1,7 @@
 # Unsealed table assignment creates an optional property
 
+**Status**: Implemented
+
 ## Summary
 
 In Luau, tables have a state, which can, among others, be "unsealed".

rfcs/unsealed-table-literals.md

@@ -1,5 +1,7 @@
 # Unsealed table literals
 
+**Status**: Implemented
+
 ## Summary
 
 Currently the only way to create an unsealed table is as an empty table literal `{}`.
@@ -73,4 +75,4 @@ We could introduce a new table state for unsealed-but-precise
 tables. The trade-off is that that would be more precise, at the cost
 of adding user-visible complexity to the type system.
 
-We could continue to treat array-like tables as sealed.
+We could continue to treat array-like tables as sealed.