* Fixed gcc warning about uninitialized `std::optional`
* Fixed inlining of functions when they are used to compute their own
arguments
In the new type solver:
* Type families that are not part of a function signature cannot be
resolved at instantiation time and will now produce an error. This will
be relaxed in the future when we get constraint clauses on function
signatures (internally)
* `never` type is now comparable
* Improved typechecking of `for..in` statements
* Fixed checks for number type in `Add` type family
* Performance was improved, with particularly large gains on large
projects
And in native code generation (jit):
* We eliminated the call instruction overhead when native code support
is enabled in the VM
* Small optimizations to arm64 lowering
* Reworked LOP_GETIMPORT handling to reduce assembly code size
* Fixed non-deterministic binary output
* Fixed bad code generation caused by incorrect SSA to VM register links
invalidation
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
* `Luau.Analyze.CLI` now has experimental support for concurrent type
checking. Use the option `-jN` where `N` is the number of threads to
spawn.
* Improve typechecking performance by ~17% by making the function
`Luau::follow` much more efficient.
* Tighten up the type of `os.date`
* Removed `ParseOptions::allowTypeAnnotations` and
`ParseOptions::supportContinueStatement`
New solver
* Improve the reliability of function overload resolution
* More work toward supporting parallel type checking
* Fix a bug in inference of `==` and `~=` which would erroneously infer
that the operands were `boolean`
* Better error reporting when `for...in` loops are used incorrectly.
CodeGen
* Fix unwind registration when libunwind is used on Linux
* Fixed replaced IR instruction use count
* Convert X64 unwind info generation to standard prologue
* Implement A64 unwind info support for Dwarf2
* Live in/out data for linear blocks is now created
* Add side-exit VM register requirements to the IR dump
* Reuse ConstPropState between block chains
* Remove redundant base update
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Added a limit on how many instructions the Compiler can safely produce
(reported by @TheGreatSageEqualToHeaven)
C++ API Changes:
* With work started on read-only and write-only properties,
`Property::type` member variable has been replaced with `TypeId type()`
and `setType(TypeId)` functions.
* New `LazyType` unwrap callback now has a `void` return type, all
that's required from the callback is to write into `unwrapped` field.
In our work on the new type solver, the following issues were fixed:
* Work has started to support https://github.com/Roblox/luau/pull/77 and
https://github.com/Roblox/luau/pull/79
* Refinements are no longer applied on l-values, removing some
false-positive errors
* Improved overload resolution against expected result type
* `Frontend::prepareModuleScope` now works in the new solver
* Cofinite strings are now comparable
And these are the changes in native code generation (JIT):
* Fixed MIN_NUM and MAX_NUM constant fold when one of the arguments is
NaN
* Added constant folding for number conversions and bit operations
* Value spilling and rematerialization is now supported on arm64
* Improved FASTCALL2K IR generation to support second argument constant
* Added value numbering and load/store propagation optimizations
* Added STORE_VECTOR on arm64, completing the IR lowering on this target
Once again, all of our changes this week are for new type solver and the
JIT.
In the new type solver, we fixed cyclic type alias handling and multiple
stability issues.
In the JIT, our main progress was for arm64, where, after lowering 36%
of instructions, we start seeing first Luau functions executing
natively.
For x64, we performed code cleanup and refactoring to allow for future
optimizations.
* A small subset of control-flow refinements have been added to
recognize type options that are unreachable after a
conditional/unconditional code block. (Fixes
https://github.com/Roblox/luau/issues/356).
Some examples:
```lua
local function f(x: string?)
if not x then return end
-- x is 'string' here
end
```
Throwing calls like `error` or `assert(false)` instead of 'return' are
also recognized.
Existing complex refinements like type/typeof and tagged union checks
are expected to work, among others.
To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to
be removed from `ExperimentalFlags.h`.
If will become enabled unconditionally in the near future.
* Linter has been integrated into the typechecker analysis so that
type-aware lint warnings can work in any mode
`Frontend::lint` methods were deprecated, `Frontend::check` has to be
used instead with `runLintChecks` option set.
Resulting lint warning are located inside `CheckResult`.
* Fixed large performance drop and increased memory consumption when
array is filled at an offset (Fixes
https://github.com/Roblox/luau/issues/590)
* Part of [Type error suppression
RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md)
was implemented making subtyping checks with `any` type transitive.
---
In our work on the new type-solver:
* `--!nocheck` mode no longer reports type errors
* New solver will not be used for `--!nonstrict` modules until all
issues with strict mode typechecking are fixed
* Added control-flow aware type refinements mentioned earlier
In native code generation:
* `LOP_NAMECALL` has been translated to IR
* `type` and `typeof` builtin fastcalls have been translated to
IR/assembly
* Additional steps were taken towards arm64 support
* Fixed incorrect lexeme generated for string parts in the middle of an
interpolated string (Fixes https://github.com/Roblox/luau/issues/744)
* DeprecatedApi lint can report some issues without type inference
information
* Fixed performance of autocomplete requests when suggestions have large
intersection types (Solves
https://github.com/Roblox/luau/discussions/847)
* Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC:
Deprecate
table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md))
* With -O2 optimization level, we now optimize builtin calls based on
known argument/return count.
Note that this change can be observable if `getfenv/setfenv` is used to
substitute a builtin, especially if arity is different.
Fastcall heavy tests show a 1-2% improvement.
* Luau can now be built with clang-cl (Fixes
https://github.com/Roblox/luau/issues/736)
We also made many improvements to our experimental components.
For our new type solver:
* Overhauled data flow analysis system, fixed issues with 'repeat'
loops, global variables and type annotations
* Type refinements now work on generic table indexing with a string
literal
* Type refinements will properly track potentially 'nil' values (like
t[x] for a missing key) and their further refinements
* Internal top table type is now isomorphic to `{}` which fixes issues
when `typeof(v) == 'table'` type refinement is handled
* References to non-existent types in type annotations no longer resolve
to 'error' type like in old solver
* Improved handling of class unions in property access expressions
* Fixed default type packs
* Unsealed tables can now have metatables
* Restored expected types for function arguments
And for native code generation:
* Added min and max IR instructions mapping to vminsd/vmaxsd on x64
* We now speculatively extract direct execution fast-paths based on
expected types of expressions which provides better optimization
opportunities inside a single basic block
* Translated existing math fastcalls to IR form to improve tag guard
removal and constant propagation
We've made a few small changes to reduce the amount of stack we use when
typechecking nested method calls (eg `foo:bar():baz():quux()`).
We've also fixed a small bytecode compiler issue that caused us to emit
redundant jump instructions in code that conditionally uses `break` or
`continue`.
On the new solver, we've switched to a new, better way to handle
augmentations to unsealed tables. We've also made some substantial
improvements to type inference and error reporting on function calls.
These things should both be on par with the old solver now.
The main improvements to the native code generator have been elimination
of some redundant type tag checks. Also, we are starting to inline
particular fastcalls directly to IR.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed rare use-after-free in analysis during table unification
A lot of work these past months went into two new Luau components:
* A near full rewrite of the typechecker using a new deferred constraint
resolution system
* Native code generation for AoT/JiT compilation of VM bytecode into x64
(avx)/arm64 instructions
Both of these components are far from finished and we don't provide
documentation on building and using them at this point.
However, curious community members expressed interest in learning about
changes that go into these components each week, so we are now listing
them here in the 'sync' pull request descriptions.
---
New typechecker can be enabled by setting
DebugLuauDeferredConstraintResolution flag to 'true'.
It is considered unstable right now, so try it at your own risk.
Even though it already provides better type inference than the current
one in some cases, our main goal right now is to reach feature parity
with current typechecker.
Features which improve over the capabilities of the current typechecker
are marked as '(NEW)'.
Changes to new typechecker:
* Regular for loop index and parameters are now typechecked
* Invalid type annotations on local variables are ignored to improve
autocomplete
* Fixed missing autocomplete type suggestions for function arguments
* Type reduction is now performed to produce simpler types to be
presented to the user (error messages, custom LSPs)
* Internally, complex types like '((number | string) & ~(false?)) |
string' can be produced, which is just 'string | number' when simplified
* Fixed spots where support for unknown and never types was missing
* (NEW) Length operator '#' is now valid to use on top table type, this
type comes up when doing typeof(x) == "table" guards and isn't available
in current typechecker
---
Changes to native code generation:
* Additional math library fast calls are now lowered to x64: math.ldexp,
math.round, math.frexp, math.modf, math.sign and math.clamp
* Fix autocompletion of if-then-else expressions
* Fix a potential crash surrounding improper use of `%*` in a string
format specifier
* All Python scripts now invoke Python via `python3` rather than
`python`.
* Improved error handling for string interpolation with too many
arguments.
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed garbage data in module scopes when type graph is not retained
* LOP_MOVE with the same source and target registers is no longer
generated (Fixes https://github.com/Roblox/luau/issues/793)
* Fixed unions of `nil` types displaying as `?`
* Internal normalization now handles class types which can make
previously failing (incorrectly) sub-typing checks to succeed
* Type mismatch errors now mention if unification failed in covariant or
invariant context, to explain why sometimes derived class can't be
converted to base class or why `T` can't be converted into `T?` and so
on
* Class type indexing is no longer an error in non-strict mode (still an
error in strict mode)
* Fixed cyclic type packs not being displayed in the type
* Added an error when unrelated types are compared with `==`/`~=`
* Fixed false positive errors involving sub-type tests an `never` type
* Fixed miscompilation of multiple assignment statements (Fixes
https://github.com/Roblox/luau/issues/754)
* Type inference stability improvements
* https://github.com/Roblox/luau/pull/719
* Improved `Failed to unify type packs` error message to be reported as
`Type pack 'X' could not be converted into 'Y'`
* https://github.com/Roblox/luau/pull/722
* 1% reduction in executed instruction count by removing a check in fast
call dispatch
* Additional fixes to reported error location of OOM errors in VM
* Improve `math.sqrt`, `math.floor` and `math.ceil` performance on
additional compilers and platforms (1-2% geomean improvement including
8-9% on math-cordic)
* All thrown exceptions by Luau analysis are derived from
`Luau::InternalCompilerError`
* When a call site has fewer arguments than required, error now reports
the location of the function name instead of the argument to the
function
* https://github.com/Roblox/luau/pull/724
* Fixed https://github.com/Roblox/luau/issues/725
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
* Support `["prop"]` syntax on class definitions in definition files.
(#704)
* Improve type checking performance for complex overloaded functions
* Fix rare cases of incorrect stack traces for out of memory errors at
runtime
* Reoptimized math.min/max/bit32 builtins assuming at least 2 arguments are used (1-2% lift on some benchmarks)
* Type errors that mention function types no longer have redundant parenthesis around return type
* Luau REPL now supports --compile=remarks which displays the source code with optimization remarks embedded as comments
* Builtin calls are slightly faster when called with 1-2 arguments (~1% improvement in some benchmarks)
- Type aliases can no longer override primitive types; attempts to do
that will result in a type error
- Fix misleading type error messages for mismatches in expression list
length during assignment
- Fix incorrect type name display in certain cases
- setmetatable/getmetatable are now ~2x faster
- tools/perfstat.py can be used to display statistics about profiles
captured via --profile switch
- Remove type definitions of
`utf8.nfcnormalize`/`nfdnormalize`/`graphemes` that aren't supported by
standalone Luau library
- Add `lua_costatus` to retrieve extended thread status (similar to
`coroutine.status`)
- Improve GC sweeping performance (2-10% improvement on allocation-heavy
benchmarks)
- Improve ComparisonPrecedence lint suggestions for three-way comparisons (X < Y < Z)
- Improve type checking stability
- Improve location information for errors when parsing invalid type annotations
- Compiler now generates bytecode version 3 in all configurations
- Improve performance of comparisons against numeric constants on AArch64
- Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative
- Fix an off-by-one error in type error text for incorrect use of string.format
- Reduce stack consumption further during parsing, hopefully eliminating stack overflows during parsing/compilation for good
- Mark interpolated string support as experimental (requires --fflags=LuauInterpolatedStringBaseSupport to enable)
- Simplify garbage collection treatment of upvalues, reducing cache misses during sweeping stage and reducing the cost of upvalue assignment (SETUPVAL); supersedes #643
- Simplify garbage collection treatment of sleeping threads
- Simplify sweeping of alive threads, reducing cache misses during sweeping stage
- Simplify management of string buffers, removing redundant linked list operations
Implements the string interpolation RFC (#165).
Adds the string interpolation as per the RFC.
```lua
local name = "world"
print(`Hello {name}!`) -- Hello world!
```
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com>
- Fix autocomplete not suggesting globals defined after the cursor (fixes#622)
- Improve type checker stability
- Reduce parser C stack consumption which fixes some stack overflow crashes on deeply nested sources
- Improve performance of bit32.extract/replace when width is implied (~3% faster chess)
- Improve performance of bit32.extract when field/width are constants (~10% faster base64)
- Heap dump now annotates thread stacks with local variable/function names
* Fix free Luau type being fully overwritten by 'any' and causing UAF
* Fix lua_clonefunction implementation replacing top instead of pushing
* Falsey values other than false can now narrow refinements
* Fix lua_getmetatable, lua_getfenv not waking thread up
* FIx a case where lua_objlen could push a new string without thread wakeup or GC
* Moved Luau math and bit32 definitions to definition file
* Improve Luau parse recovery of incorrect return type token
* Adds a currently unused x86-64 assembler as a prerequisite for possible future JIT compilation
* Fix a bug in table iteration (closes Possible table iteration bug #504)
* Improved warning method when function is used as a type
* Fix a bug with unsandboxed iteration with pairs()
* Type of coroutine.status() is now a union of value types
* Bytecode output for tests/debugging now has labels
* Improvements to loop unrolling cost estimation
* Report errors when the key obviously doesn't exist in the table
- Improve refinement support for unions, in particular it's now possible to implement tagged unions as a union of tables where individual branches use a string literal type for one of the fields.
- Fix `string.split` type information
- Optimize `select(_, ...)` to run in constant time (~2.7x faster on VariadicSelect benchmark)
- Improve debug line information for multi-line assignments
- Improve compilation of table literals when table keys are constant expressions/variables
- Use forward GC barrier for `setmetatable` which slightly accelerates GC progress
- TableOperations lint now includes a warning for table.create(N, {}) (which is likely a mistake since the table is shared by all entries)
- Type checker now type checks #v when v is a union
- Parser now rejects sources that consists of a single unfinished long comment
- Work around significant MSVC 2022 performance regression, bringing it more or less in line with MSVC 2019
- Compiler now predicts array size for newly allocated tables when the table is filled in a short loop
- Small improvements in compilation throughput (~2% faster)
- Implement paged sweeper for GC which improves sweep throughput 2-3x and reduces memory consumption by 8 bytes per object (once it is stabilized we will see additional 8 bytes per object of savings)
- Improve Repl Tab completion
- Repl now supports -i (interactive mode to run code in context of a script's environment) and -On (to control optimization flags)
- Rework transaction log used for type checking which should result in more robust type checking internals with fewer bugs
- Reduce the amount of memory consumed by type checker on large module graphs
- Type checker now errors on attempts to change the type of imported module fields
- The return type of newproxy is now any (fixes#296)
- Implement new number printing algorithm (Schubfach) which makes tostring() produce precise (round-trippable) and short decimal output up to 10x faster
- Fix lua_Debug::linedefined to point to the line with the function definition instead of the first statement (fixes#265)
- Fix minor bugs in Tab completion in Repl
- Repl now saves/restores command history in ~/.luau_history
This doesn't contain all changes for 507 yet but we might want to do the
Luau 0.507 release a bit earlier to end the year sooner.
Changes:
- Type ascription (::) now permits casts between related types in both directions, allowing to refine or loosen the type (RFC #56)
- Fix type definition for tonumber to return number? since the input string isn't guaranteed to contain a valid number
- Fix type refinements for field access via []
- Many stability fixes for type checker
- Provide extra information in error messages for type mismatches in more cases
- Improve performance of type checking for large unions when union members are string literals
- Add coverage reporting support to Repl (--coverage command line argument) and lua_getcoverage C API
- Work around code signing issues during Makefile builds on macOS
- Improve performance of truthiness checks in some cases, particularly on Apple M1, resulting in 10-25% perf gains on qsort benchmark depending on the CPU/compiler
- Fix support for little-endian systems; IBM s390x here we go!
