With this optimization, built-in vector constructor calls with 3/4 arguments are detected by the compiler and turned into vector constants when the arguments are constant numbers.
Requires optimization level 2 because built-ins are not folded otherwise by the compiler.
Bytecode version is bumped because of the new constant type, but old bytecode versions can still be loaded.
The following synthetic benchmark shows ~6.6x improvement.
```
local v
for i = 1, 10000000 do
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
v = vector(1, 2, 3)
end
```
Also tried a more real world scenario and could see a few percent improvement.
Added a new fast flag LuauVectorLiterals for enabling the feature.
---------
Co-authored-by: Petri Häkkinen <petrih@rmd.remedy.fi>
Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
# What's changed?
- Record the location of properties for table types (closes#802)
- Implement stricter UTF-8 validations as per the RFC
(https://github.com/luau-lang/rfcs/pull/1)
- Implement `buffer` as a new type in both the old and new solvers.
- Changed errors produced by some `buffer` builtins to be a bit more
generic to avoid platform-dependent error messages.
- Fixed a bug where `Unifier` would copy some persistent types, tripping
some internal assertions.
- Type checking rules on relational operators is now a little bit more
lax.
- Improve dead code elimination for some `if` statements with complex
always-false conditions
## New type solver
- Dataflow analysis now generates phi nodes on exit of branches.
- Dataflow analysis avoids producing a new definition for locals or
properties that are not owned by that loop.
- If a function parameter has been constrained to `never`, report errors
at all uses of that parameter within that function.
- Switch to using the new `Luau::Set` to replace `std::unordered_set` to
alleviate some poor allocation characteristics which was negatively
affecting overall performance.
- Subtyping can now report many failing reasons instead of just the
first one that we happened to find during the test.
- Subtyping now also report reasons for type pack mismatches.
- When visiting `if` statements or expressions, the resulting context
are the common terms in both branches.
## Native codegen
- Implement support for `buffer` builtins to its IR for x64 and A64.
- Optimized `table.insert` by not inserting a table barrier if it is
fastcalled with a constant.
## Internal Contributors
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Arseny Kapoulkine <arseny@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
# What's changed?
* Fixed a bug in type cloning by maintaining persistent types.
* We now parse imprecise integer literals to report the imprecision as a
warning to developers.
* Add a compiler flag to specify the name of the statistics output file.
### New type solver
* Renamed `ConstraintGraphBuilder` to `ConstraintGenerator`
* LValues now take into account the type being assigned during
constraint generation.
* Normalization performance has been improved by 33% by replacing the an
internal usage of `std::unordered_set` with `DenseHashMap`.
* Normalization now has a helper to identify types that are equivalent
to `unknown`, which is being used to fix some bugs in subtyping.
* Uses of the old unifier in the new type solver have been eliminated.
* Improved error explanations for subtyping errors in `TypeChecker2`.
### Native code generation
* Expanded some of the statistics recorded during compilation to include
the number of instructions and blocks.
* Introduce instruction and block count limiters for controlling what
bytecode is translated into native code.
* Implement code generation for byteswap instruction.
### Internal Contributors
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
## What's changed
- `bit32.byteswap` added
([RFC](4f543ec23b/docs/function-bit32-byteswap.md))
- Buffer library implementation
([RFC](4f543ec23b/docs/type-byte-buffer.md))
- Fixed a missing `stdint.h` include
- Fixed parser limiter for recursive type annotations being kind of
weird (fixes#645)
### Native Codegen
- Fixed a pair of issues when lowering `bit32.extract`
- Fixed a narrow edge case that could result in an infinite loop without
an interruption
- Fixed a negative array out-of-bounds access issue
- Temporarily reverted linear block predecessor value propagation
### New type solver
- We now type check assignments to annotated variables
- Fixed some test cases under local type inference
- Moved `isPending` checks for type families to improve performance
- Optimized our process for testing if a free type is sufficiently
solved
- Removed "none ptr" from lea instruction disassembly logging
### Build system & tooling
- CMake configuration now validates dependencies to maintain separation
between components
- Improvements to the fuzzer coverage
- Deduplicator for fuzzed callstacks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
I've decided to take a stab at implementing `bit32.byteswap` from the
[recently merged
RFC](https://github.com/Roblox/luau/blob/master/rfcs/function-bit32-byteswap.md).
I asked on Discord for some guidance, but for the sake of posterity:
this is my first time doing this and I am likely to have made some
mistakes.
The biggest gaps in this implementation are the lack of tests and the
lack of native codegen support. I'd appreciate help with those since I'm
not sure what's relevant for me to touch for tests, and I'm told that
relevant assembler instructions don't exist publicly yet. Intuition
tells me that Luau-side tests would go into
`tests/conformance/bitwise.luau` but this is not well documented and I'm
not sure how I'm meant to test built-in implementations.
The current implementation compiles down to `bswap` and `rev` on x86 and
ARM respectively when optimized.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
### What's Changed
- Improve readability of unions and intersections by limiting the number
of elements of those types that can be presented on a single line (gated
under `FFlag::LuauToStringSimpleCompositeTypesSingleLine`)
- Adds a new option to the compiler `--record-stats` to record and
output compilation statistics
- `if...then...else` expressions are now optimized into `AND/OR` form
when possible.
### VM
- Add a new `buffer` type to Luau based on the [buffer
RFC](https://github.com/Roblox/luau/pull/739) and additional C API
functions to work with it; this release does not include the library.
- Internal C API to work with string buffers has been updated to align
with Lua version more closely
### Native Codegen
- Added support for new X64 instruction (rev) and new A64 instruction
(bswap) in the assembler
- Simplified the way numerical loop condition is translated to IR
### New Type Solver
- Operator inference now handled by type families
- Created a new system called `Type Paths` to explain why subtyping
tests fail in order to improve the quality of error messages.
- Systematic changes to implement Data Flow analysis in the new solver
(`Breadcrumb` removed and replaced with `RefinementKey`)
---
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
* Include `windows.h` rather than `Windows.h` to make things compile on
MinGW.
* Custom implementation of timegm/os.time for all platforms
* Disable builtin constant folding when getfenv/setfenv are used
* Fixes https://github.com/Roblox/luau/issues/1042
* Fixes https://github.com/Roblox/luau/issues/1043
New Type Checker
* Initial work toward type states.
* Rework most overloadable operators to use type families.
* Initial work toward our new nonstrict mode.
Native Codegen
* Fix native code generation for dead loops
* Annotate top-level functions as cold
* Slightly smaller/faster x64 Luau calls
* emitInstCall used to not set savedpc itself, but now it does for
consistency with all other implementations
* Implement cmov support for X64
* Fix assertion in luau-compile when module is empty
* Optimize A64 calls at some code size cost
* Inline constant array index offset into the load/store instruction
* Increase x64 spill slots from 5 to 13
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
# New Type Solver
- Implement bidirectional type inference for higher order functions so
that we can provide a more precise type improving the autocomplete's
human factors.
- We seal all tables, so we changed the stringification to make it a
little lighter on users.
- Fixed a case of array-out-of-bound access.
- Type families no longer depends on `TxnLog` and `Unifier`.
- Type refinements now waits until the free types are sufficiently
solved.
# Native Code Generation
- Remove cached slot lookup for `executeSETTABLEKS` function because it
is a fallback in the event of a cache miss, making the cached slot
lookup redundant.
- Optimized repeated array lookups, e.g. `a[3]` in `a[3] = a[3] / 2` is
done once.
# Misc
- On some platforms, it is necessary to use `gmtime_s` with the
arguments reversed to get the current time. You can now define
`DOCTEST_CONFIG_USE_GMTIME_S` to build and run unit tests on those
platforms.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
- Cleaned up `FFlag::FixFindBindingAtFunctionName`,
`FFlag::LuauNormalizeBlockedTypes`, `FFlag::LuauPCallDebuggerFix`
- Added support for break and continue into control flow analysis
- The old type unification engine will now report a more fine-grained
error at times, indicating that type normalization in particular failed
# New Type Solver
- Refactor of Unifier2, the new unification implementation for Luau
- Completed MVP of new unification implementation
- Dramatically simplified overload selection logic
- Type family reduction can now apply sooner to free types that have
been solved
- Subtyping now supports table indexers
- Generalization now replaces bad generics with unknown
# Native Code Generation
- Reduce stack spills caused by FINDUPVAL and STORE_TAG
- Improve Generate SHL/SHR/SAR/rotates with immediate operands in X64
- Removed redundant case re-check in table lookup fallback
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
* Rerun clang-format on the code
* Fix the variance on indexer result subtyping. This fixes some issues
with inconsistent error reporting.
* Fix a bug in the normalization logic for intersections of strings
New Type Solver
* New overload selection logic
* Subtype tests now correctly treat a generic as its upper bound within
that generic's scope
* Semantic subtyping for negation types
* Semantic subtyping between strings and compatible table types like
`{lower: (string) -> string}`
* Further work toward finalizing our new subtype test
* Correctly generalize module-scope symbols
Native Codegen
* Lowering statistics for assembly
* Make executable allocation size/limit configurable without a rebuild.
Use `FInt::LuauCodeGenBlockSize` and `FInt::LuauCodeGenMaxTotalSize`.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
* Fixed `Frontend::markDirty` not working on modules that were not
typechecked yet
* Fixed generic variadic function unification succeeding when it should
have reported an error
New Type Solver:
* Implemented semantic subtyping check for function types
Native Code Generation:
* Improved performance of numerical loops with a constant step
* Simplified IR for `bit32.extract` calls extracting first/last bits
* Improved performance of NaN checks
CLI/Reduce.cpp:
Never worked under MinGW as-is. Add check for MinGW as they're defined.
VM/src/lmem.cpp:
MinGW layout follows more closely MSVC. Checks before were only correct
for 32-bit gcc / clang in non-Windows.
NOTES:
__MINGW32__ is defined on both 32-bit and 64-bit, __MINGW64__ is 64-bit
only. 32-bit MinGW compilers (both Clang & GCC) have a floating point
test error on math.noise, specifically math.lua:258. All other test
cases / unit tests pass modulo stack size issues (so requires optimized build).
---------
Co-authored-by: jdp_ <42700985+jdpatdiscord@users.noreply.github.com>
- Updated Roblox copyright to 2023
- Floor division operator `//` (implements #832)
- Autocomplete now shows `end` within `do` blocks
- Restore BraceType when using `Lexer::lookahead` (fixes#1019)
# New typechecker
- Subtyping tests between metatables and tables
- Subtyping tests between string singletons and tables
- Subtyping tests for class types
# Native codegen
- Fixed macOS test failure (wrong spill restore offset)
- Fixed clobbering of non-volatile xmm registers on Windows
- Fixed wrong storage location of SSA reg spills
- Implemented A64 support for add/sub extended
- Eliminated zextReg from A64 lowering
- Remove identical table slot lookups
- Propagate values from predecessor into the linear block
- Disabled reuse slot optimization
- Keep `LuaNode::val` check for nil when optimizing `CHECK_SLOT_MATCH`
- Implemented IR translation of `table.insert` builtin
- Fixed mmap error handling on macOS/Linux
# Tooling
- Used `|` as a column separator instead of `+` in `bench.py`
- Added a `table.sort` micro-benchmark
- Switched `libprotobuf-mutator` to a less problematic version
* AST queries at position where function name is will now return
AstExprLocal
* Lexer performance has been slightly improved
* Fixed incorrect string singleton autocomplete suggestions (fixes#858)
* Improved parsing error messages
* Fixed crash on null pointer access in unification (fixes#1017)
* Native code support is enabled by default and `native=1`
(make)/`LUAU_NATIVE` (CMake)/`-DLUA_CUSTOM_EXECUTION` configuration is
no longer required
New typechecker:
* New subtyping check can now handle generic functions and tables
(including those that contain cycles)
Native code generation:
* Loops with non-numeric parameters are now handled by VM to streamline
native code
* Array size check can be optimized away in SETLIST
* On failure, CodeGen::compile returns a reason
* Fixed clobbering of non-volatile xmm registers on Windows
* Fix a use-after-free bug in the new type cloning algorithm
* Tighten up the type of `coroutine.wrap`. It is now `<A..., R...>(f:
(A...) -> R...) -> ((A...) -> R...)`
* Break `.luaurc` out into a separate library target `Luau.Config`. This
makes it easier for applications to reason about config files without
also depending on the type inference engine.
* Move typechecking limits into `FrontendOptions`. This allows embedders
more finely-grained control over autocomplete's internal time limits.
* Fix stability issue with debugger onprotectederror callback allowing
break in non-yieldable contexts
New solver:
* Initial work toward [Local Type
Inference](0e1082108f/rfcs/local-type-inference.md)
* Introduce a new subtyping test. This will be much nicer than the old
test because it is completely separate both from actual type inference
and from error reporting.
Native code generation:
* Added function to compute iterated dominance frontier
* Optimize barriers in SET_UPVALUE when tag is known
* Cache lua_State::global in a register on A64
* Optimize constant stores in A64 lowering
* Track table array size state to optimize array size checks
* Add split tag/value store into a VM register
* Check that spills can outlive the block only in specific conditions
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Better indentation in multi-line type mismatch error messages
* Error message clone can no longer cause a stack overflow (when
typechecking with retainFullTypeGraphs set to false); fixes
https://github.com/Roblox/luau/issues/975
* `string.format` with %s is now ~2x faster on strings smaller than 100
characters
Native code generation:
* All VM side exits will block return to the native execution of the
current function to preserve correctness
* Fixed executable page allocation on Apple platforms when using
hardened runtime
* Added statistics for code generation (no. of functions compiler,
memory used for different areas)
* Fixed issue with function entry type checks performed more that once
in some functions
* Progress toward a diffing algorithm for types. We hope that this will
be useful for writing clearer error messages.
* Add a missing recursion limiter in `Unifier::tryUnifyTables`. This was
causing a crash in certain situations.
* Luau heap graph enumeration improvements:
* Weak references are not reported
* Added tag as a fallback name of non-string table links
* Included top Luau function information in thread name to understand
where thread might be suspended
* Constant folding for `math.pi` and `math.huge` at -O2
* Optimize `string.format` and `%*`
* This change makes string interpolation 1.5x-2x faster depending on the
number and type of formatted components, assuming a few are using
primitive types, and reduces associated GC pressure.
New type checker:
* Initial work toward tracking the upper and lower bounds of types
accurately.
Native code generation (JIT):
* Add IrCmd::CHECK_TRUTHY for improved assert fast-calls
* Do not compute type map for modules without types
* Capture metatable+readonly state for NEW_TABLE IR instructions
* Replace JUMP_CMP_ANY with CMP_ANY and existing JUMP_EQ_INT
* Add support for exits to VM with reentry lock in VmExit
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Type checker/autocomplete:
* `Luau::autocomplete` no longer performs typechecking internally, make
sure to run `Frontend::check` before performing autocomplete requests
* Autocomplete string suggestions without "" are now only suggested
inside the ""
* Autocomplete suggestions now include `function (anonymous autofilled)`
key with a full suggestion for the function expression (with arguments
included) stored in `AutocompleteEntry::insertText`
* `AutocompleteEntry::indexedWithSelf` is provided for function call
suggestions made with `:`
* Cyclic modules now see each other type exports as `any` to prevent
memory use-after-free (similar to module return type)
Runtime:
* Updated inline/loop unroll cost model to better handle assignments
(Fixes https://github.com/Roblox/luau/issues/978)
* `math.noise` speed was improved by ~30%
* `table.concat` speed was improved by ~5-7%
* `tonumber` and `tostring` now have fastcall paths that execute ~1.5x
and ~2.5x faster respectively (fixes#777)
* Fixed crash in `luaL_typename` when index refers to a non-existing
value
* Fixed potential out of memory scenario when using `string.sub` or
`string.char` in a loop
* Fixed behavior of some fastcall builtins when called without arguments
under -O2 to match original functions
* Support for native code execution in VM is now enabled by default
(note: native code still has to be generated explicitly)
* `Codegen::compile` now accepts `CodeGen_OnlyNativeModules` flag. When
set, only modules that have a `--!native` hot-comment at the top will be
compiled to native code
In our new typechecker:
* Generic type packs are no longer considered to be variadic during
unification
* Timeout and cancellation now works in new solver
* Fixed false positive errors around 'table' and 'function' type
refinements
* Table literals now use covariant unification rules. This is sound
since literal has no type specified and has no aliases
* Fixed issues with blocked types escaping the constraint solver
* Fixed more places where error messages that should've been suppressed
were still reported
* Fixed errors when iterating over a top table type
In our native code generation (jit):
* 'DebugLuauAbortingChecks' flag is now supported on A64
* LOP_NEWCLOSURE has been translated to IR
* Added support for async typechecking cancellation using a token passed
through frontend options
* Added luaC_enumheap for building debug tools that need a graph of Luau
heap
In our new typechecker:
* Errors or now suppressed when checking property lookup of
error-suppressing unions
In our native code generation (jit):
* Fixed unhandled value type in NOT_ANY lowering
* Fast-call tag checks will exit to VM on failure, instead of relying on
a native fallback
* Added vector type to the type information
* Eliminated redundant direct jumps across dead blocks
* Debugger APIs are now disabled for call frames executing natively
* Implemented support for unwind registration on macOS 14
* Fixed indexing table intersections using `x["prop"]` syntax:
https://github.com/Roblox/luau/pull/971
* Add console output codepage for Windows:
https://github.com/Roblox/luau/pull/967
* Added `Frontend::parse` for a fast source graph preparation
* luau_load should check GC
* Work toward a type-diff system for nicer error messages
New Solver
* Correctly suppress errors in more cases
* Further improvements to typechecking of function calls and return
statements
* Crash fixes
* Propagate refinements drawn from the condition of a while loop into
the loop body
JIT
* Fix accidental bailout for math.frexp/modf/sign in A64
* Work toward bringing type annotation info in
* Do not propagate Luau IR constants of wrong type into load
instructions
* CHECK_SAFEENV exits to VM on failure
* Implement error handling in A64 reg allocator
* Inline the string.len builtin
* Do not enter native code of a function if arguments don’t match
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Definition files can now ascribe indexers to class types.
(https://github.com/Roblox/luau/pull/949)
* Remove --compile support from the REPL. You can just use luau-compile
instead.
* When an exception is thrown during parallel typechecking (usually an
ICE), we now gracefully stop typechecking and drain active workers
before rethrowing the exception.
New solver
* Include more source location information when we hit an internal
compiler error
* Improve the logic that simplifies intersections of tables
JIT
* Save testable type annotations to bytecode
* Improve block placement for linearized blocks
* Add support for lea reg, [rip+offset] for labels
* Unify X64 and A64 codegen for RETURN
* Outline interrupt handlers for X64
* Remove global rArgN in favor of build.abi
* Change A64 INTERRUPT lowering to match X64
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Added luau-compile executable target to build/test compilation without
having full REPL included.
In our new typechecker:
* Fixed the order in which constraints are checked to get more
deterministic errors in different environments
* Fixed `isNumber`/`isString` checks to fix false positive errors in
binary comparisons
* CannotCallNonFunction error is reported when calling an intersection
type of non-functions
In our native code generation (jit):
* Outlined X64 return instruction code to improve code size
* Improved performance of return instruction on A64
* Added construction of a dominator tree for future optimizations
A pretty small changelist this week:
* When type inference fails to find any matching overload for a
function, we were declining to commit any changes to the type graph at
all. This was resulting in confusing type errors in certain cases. Now,
when a matching overload cannot be found, we always commit to the first
overload we tried.
JIT
* Fix missing variadic register invalidation in FALLBACK_GETVARARGS
* Add a missing null pointer check for the result of luaT_gettm
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed gcc warning about uninitialized `std::optional`
* Fixed inlining of functions when they are used to compute their own
arguments
In the new type solver:
* Type families that are not part of a function signature cannot be
resolved at instantiation time and will now produce an error. This will
be relaxed in the future when we get constraint clauses on function
signatures (internally)
* `never` type is now comparable
* Improved typechecking of `for..in` statements
* Fixed checks for number type in `Add` type family
* Performance was improved, with particularly large gains on large
projects
And in native code generation (jit):
* We eliminated the call instruction overhead when native code support
is enabled in the VM
* Small optimizations to arm64 lowering
* Reworked LOP_GETIMPORT handling to reduce assembly code size
* Fixed non-deterministic binary output
* Fixed bad code generation caused by incorrect SSA to VM register links
invalidation
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Lots of things going on this week:
* Fix a crash that could occur in the presence of a cyclic union. We
shouldn't be creating cyclic unions, but we shouldn't be crashing when
they arise either.
* Minor cleanup of `luau_precall`
* Internal change to make L->top handling slightly more uniform
* Optimize SETGLOBAL & GETGLOBAL fallback C functions.
* https://github.com/Roblox/luau/pull/929
* The syntax to the `luau-reduce` commandline tool has changed. It now
accepts a script, a command to execute, and an error to search for. It
no longer automatically passes the script to the command which makes it
a lot more flexible. Also be warned that it edits the script it is
passed **in place**. Do not point it at something that is not in source
control!
New solver
* Switch to a greedier but more fallible algorithm for simplifying union
and intersection types that are created as part of refinement
calculation. This has much better and more predictable performance.
* Fix a constraint cycle in recursive function calls.
* Much improved inference of binary addition. Functions like `function
add(x, y) return x + y end` can now be inferred without annotations. We
also accurately typecheck calls to functions like this.
* Many small bugfixes surrounding things like table indexers
* Add support for indexers on class types. This was previously added to
the old solver; we now add it to the new one for feature parity.
JIT
* https://github.com/Roblox/luau/pull/931
* Fuse key.value and key.tt loads for CEHCK_SLOT_MATCH in A64
* Implement remaining aliases of BFM for A64
* Implement new callinfo flag for A64
* Add instruction simplification for int->num->int conversion chains
* Don't even load execdata for X64 calls
* Treat opcode fallbacks the same as manually written fallbacks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* `ClassType` can now have an indexer defined on it. This allows custom
types to be used in `t[x]` expressions.
* Fixed search for closest executable breakpoint line. Previously,
breakpoints might have been skipped in `else` blocks at the end of a
function
* Fixed how unification is performed for two optional types `a? <: b?`,
previously it might have unified either 'a' or 'b' with 'nil'. Note that
this fix is not enabled by default yet (see the list in
`ExperimentalFlags.h`)
In the new type solver, a concept of 'Type Families' has been
introduced.
Type families can be thought of as type aliases with custom type
inference/reduction logic included with them.
For example, we can have an `Add<T, U>` type family that will resolve
the type that is the result of adding two values together.
This will help type inference to figure out what 'T' and 'U' might be
when explicit type annotations are not provided.
In this update we don't define any type families, but they will be added
in the near future.
It is also possible for Luau embedders to define their own type families
in the global/environment scope.
Other changes include:
* Fixed scope used to find out which generic types should be included in
the function generic type list
* Fixed a crash after cyclic bound types were created during unification
And in native code generation (jit):
* Use of arm64 target on M1 now requires macOS 13
* Entry into native code has been optimized. This is especially
important for coroutine call/pcall performance as they involve going
through a C call frame
* LOP_LOADK(X) translation into IR has been improved to enable type
tag/constant propagation
* arm64 can use integer immediate values to synthesize floating-point
values
* x64 assembler removes duplicate 64bit numbers from the data section to
save space
* Linux `perf` can now be used to profile native Luau code (when running
with --codegen-perf CLI argument)
* `Luau.Analyze.CLI` now has experimental support for concurrent type
checking. Use the option `-jN` where `N` is the number of threads to
spawn.
* Improve typechecking performance by ~17% by making the function
`Luau::follow` much more efficient.
* Tighten up the type of `os.date`
* Removed `ParseOptions::allowTypeAnnotations` and
`ParseOptions::supportContinueStatement`
New solver
* Improve the reliability of function overload resolution
* More work toward supporting parallel type checking
* Fix a bug in inference of `==` and `~=` which would erroneously infer
that the operands were `boolean`
* Better error reporting when `for...in` loops are used incorrectly.
CodeGen
* Fix unwind registration when libunwind is used on Linux
* Fixed replaced IR instruction use count
* Convert X64 unwind info generation to standard prologue
* Implement A64 unwind info support for Dwarf2
* Live in/out data for linear blocks is now created
* Add side-exit VM register requirements to the IR dump
* Reuse ConstPropState between block chains
* Remove redundant base update
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Added a limit on how many instructions the Compiler can safely produce
(reported by @TheGreatSageEqualToHeaven)
C++ API Changes:
* With work started on read-only and write-only properties,
`Property::type` member variable has been replaced with `TypeId type()`
and `setType(TypeId)` functions.
* New `LazyType` unwrap callback now has a `void` return type, all
that's required from the callback is to write into `unwrapped` field.
In our work on the new type solver, the following issues were fixed:
* Work has started to support https://github.com/Roblox/luau/pull/77 and
https://github.com/Roblox/luau/pull/79
* Refinements are no longer applied on l-values, removing some
false-positive errors
* Improved overload resolution against expected result type
* `Frontend::prepareModuleScope` now works in the new solver
* Cofinite strings are now comparable
And these are the changes in native code generation (JIT):
* Fixed MIN_NUM and MAX_NUM constant fold when one of the arguments is
NaN
* Added constant folding for number conversions and bit operations
* Value spilling and rematerialization is now supported on arm64
* Improved FASTCALL2K IR generation to support second argument constant
* Added value numbering and load/store propagation optimizations
* Added STORE_VECTOR on arm64, completing the IR lowering on this target
* Fixed exported types not being suggested in autocomplete
* `T...` is now convertible to `...any` (Fixes
https://github.com/Roblox/luau/issues/767)
* Fixed issue with `T?` not being convertible to `T | T` or `T?`
(sometimes when internal pointer identity is different)
* Fixed potential crash in missing table key error suggestion to use a
similar existing key
* `lua_topointer` now returns a pointer for strings
C++ API Changes:
* `prepareModuleScope` callback has moved from TypeChecker to Frontend
* For LSPs, AstQuery functions (and `isWithinComment`) can be used
without full Frontend data
A lot of changes in our two experimental components as well.
In our work on the new type-solver, the following issues were fixed:
* Fixed table union and intersection indexing
* Correct custom type environments are now used
* Fixed issue with values of `free & number` type not accepted in
numeric operations
And these are the changes in native code generation (JIT):
* arm64 lowering is almost complete with support for 99% of IR commands
and all fastcalls
* Fixed x64 assembly encoding for extended byte registers
* More external x64 calls are aware of register allocator
* `math.min`/`math.max` with more than 2 arguments are now lowered to IR
as well
* Fixed correctness issues with `math` library calls with multiple
results in variadic context and with x64 register conflicts
* x64 register allocator learnt to restore values from VM memory instead
of always using stack spills
* x64 exception unwind information now supports multiple functions and
fixes function start offset in Dwarf2 info
Some userdata objects may need to support manual destruction in addition
to automatic GC. For example, files, threads, GPU resources and objects
with large external allocations.
With Lua, a finalizer can be _generically_ called by invoking the __gc
metamethod manually, but this is currently not possible with tagged
userdata in Luau because it's not possible to query the destructor
associated with an userdata. While it is possible to workaround this by
duplicating the destructor table locally on client side (*), it's more
convenient to deduplicate the data and get the destructor using the API
instead.
(*) Note: a separate destructor table for each VM may be required if the
VMs use different set of tags.
Implementation notes:
1. I first considered adding a typedef for lua_Destructor but
unfortunately there are two kinds of destructors, one with and one
without the lua_State* argument, so I decided against it at this point.
Maybe it should be added later if the destructor API is unified (by
dropping the Lua state pointer argument?).
2. For some reason the conformance test produced warning "qualifier
applied to function type has no meaning; ignored" on VS2017 (possibly
because the test framework does not like function pointers for some
reason?). I silenced this by pulling out the test expressions from those
CHECKs.
* `table.sort` was improved further. It now guarentees N*log(N) time
complexity in the worst case.
* Fix https://github.com/Roblox/luau/issues/880
We are also working on fixing final bugs and crashes in the new type
solver.
On the CodeGen front we have a few things going on:
* We have a smarter register allocator for the x86 JIT
* We lower more instructions on arm64
* The vector constructor builtin is now translated to IR
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Once again, all of our changes this week are for new type solver and the
JIT.
In the new type solver, we fixed cyclic type alias handling and multiple
stability issues.
In the JIT, our main progress was for arm64, where, after lowering 36%
of instructions, we start seeing first Luau functions executing
natively.
For x64, we performed code cleanup and refactoring to allow for future
optimizations.
All of our changes this week have been focused on the new type solver
and the JIT.
As we march toward feature parity with the old solver, we've tightened
up a bunch of lingering issues with overload resolution, unsealed
tables, and type normalization. We've also fixed a bunch of crashes and
assertion failures in the new solver.
On the JIT front, we've started work on an A64 backend, improved the IR
analysis in a bunch of cases, and implemented assembly generation for
the builtin functions `type()` and `typeof()`.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* A small subset of control-flow refinements have been added to
recognize type options that are unreachable after a
conditional/unconditional code block. (Fixes
https://github.com/Roblox/luau/issues/356).
Some examples:
```lua
local function f(x: string?)
if not x then return end
-- x is 'string' here
end
```
Throwing calls like `error` or `assert(false)` instead of 'return' are
also recognized.
Existing complex refinements like type/typeof and tagged union checks
are expected to work, among others.
To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to
be removed from `ExperimentalFlags.h`.
If will become enabled unconditionally in the near future.
* Linter has been integrated into the typechecker analysis so that
type-aware lint warnings can work in any mode
`Frontend::lint` methods were deprecated, `Frontend::check` has to be
used instead with `runLintChecks` option set.
Resulting lint warning are located inside `CheckResult`.
* Fixed large performance drop and increased memory consumption when
array is filled at an offset (Fixes
https://github.com/Roblox/luau/issues/590)
* Part of [Type error suppression
RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md)
was implemented making subtyping checks with `any` type transitive.
---
In our work on the new type-solver:
* `--!nocheck` mode no longer reports type errors
* New solver will not be used for `--!nonstrict` modules until all
issues with strict mode typechecking are fixed
* Added control-flow aware type refinements mentioned earlier
In native code generation:
* `LOP_NAMECALL` has been translated to IR
* `type` and `typeof` builtin fastcalls have been translated to
IR/assembly
* Additional steps were taken towards arm64 support
* Fix#817
* Fix#850
* Optimize math.floor/ceil/round with SSE4.1
* Results in a ~7-9% speedup on the math-cordic benchmark.
* Optimized table.sort.
* table.sort is now ~4.1x faster (when not using a predicate) and ~2.1x
faster when using a simple predicate. Performance may improve further in
the future.
* Reorganize the memory ownership of builtin type definitions.
* This is a small initial step toward affording parallel typechecking.
The new type solver is coming along nicely. We are working on fixing
crashes and bugs.
A few major changes to native codegen landed this week:
* Fixed lowering of Luau IR mod instruction when first argument is a
constant
* Added VM register data-flow/capture analysis
* Fixed issues with optimizations in unreachable blocks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed incorrect lexeme generated for string parts in the middle of an
interpolated string (Fixes https://github.com/Roblox/luau/issues/744)
* DeprecatedApi lint can report some issues without type inference
information
* Fixed performance of autocomplete requests when suggestions have large
intersection types (Solves
https://github.com/Roblox/luau/discussions/847)
* Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC:
Deprecate
table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md))
* With -O2 optimization level, we now optimize builtin calls based on
known argument/return count.
Note that this change can be observable if `getfenv/setfenv` is used to
substitute a builtin, especially if arity is different.
Fastcall heavy tests show a 1-2% improvement.
* Luau can now be built with clang-cl (Fixes
https://github.com/Roblox/luau/issues/736)
We also made many improvements to our experimental components.
For our new type solver:
* Overhauled data flow analysis system, fixed issues with 'repeat'
loops, global variables and type annotations
* Type refinements now work on generic table indexing with a string
literal
* Type refinements will properly track potentially 'nil' values (like
t[x] for a missing key) and their further refinements
* Internal top table type is now isomorphic to `{}` which fixes issues
when `typeof(v) == 'table'` type refinement is handled
* References to non-existent types in type annotations no longer resolve
to 'error' type like in old solver
* Improved handling of class unions in property access expressions
* Fixed default type packs
* Unsealed tables can now have metatables
* Restored expected types for function arguments
And for native code generation:
* Added min and max IR instructions mapping to vminsd/vmaxsd on x64
* We now speculatively extract direct execution fast-paths based on
expected types of expressions which provides better optimization
opportunities inside a single basic block
* Translated existing math fastcalls to IR form to improve tag guard
removal and constant propagation
* Fixed rare use-after-free in analysis during table unification
A lot of work these past months went into two new Luau components:
* A near full rewrite of the typechecker using a new deferred constraint
resolution system
* Native code generation for AoT/JiT compilation of VM bytecode into x64
(avx)/arm64 instructions
Both of these components are far from finished and we don't provide
documentation on building and using them at this point.
However, curious community members expressed interest in learning about
changes that go into these components each week, so we are now listing
them here in the 'sync' pull request descriptions.
---
New typechecker can be enabled by setting
DebugLuauDeferredConstraintResolution flag to 'true'.
It is considered unstable right now, so try it at your own risk.
Even though it already provides better type inference than the current
one in some cases, our main goal right now is to reach feature parity
with current typechecker.
Features which improve over the capabilities of the current typechecker
are marked as '(NEW)'.
Changes to new typechecker:
* Regular for loop index and parameters are now typechecked
* Invalid type annotations on local variables are ignored to improve
autocomplete
* Fixed missing autocomplete type suggestions for function arguments
* Type reduction is now performed to produce simpler types to be
presented to the user (error messages, custom LSPs)
* Internally, complex types like '((number | string) & ~(false?)) |
string' can be produced, which is just 'string | number' when simplified
* Fixed spots where support for unknown and never types was missing
* (NEW) Length operator '#' is now valid to use on top table type, this
type comes up when doing typeof(x) == "table" guards and isn't available
in current typechecker
---
Changes to native code generation:
* Additional math library fast calls are now lowered to x64: math.ldexp,
math.round, math.frexp, math.modf, math.sign and math.clamp
`lua_getupvalue()` and `lua_setupvalue()` don't behave as expected when
working with Lua closure whose `Proto` has no debug info. The code
currently uses `sizeupvalues` to do bounds checking of upvalue indices,
but that's the size of the upvalue _names_ array. It will always be `0`
if the `Proto` doesn't have debug info.
This uses `nups` instead, and just returns `""` as the upvalue name if
we don't have one, same as for C closures.
Co-authored-by: Harold Cindy <HaroldCindy@users.noreply.github.com>
* Fix autocompletion of if-then-else expressions
* Fix a potential crash surrounding improper use of `%*` in a string
format specifier
* All Python scripts now invoke Python via `python3` rather than
`python`.
* Improved error handling for string interpolation with too many
arguments.
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
This is useful in particular if redefine `LUAI_MAXCSTACK` to a higher
value than the current one (8000).
Ie. passing `-D LUAI_MAXCSTACK=1000000` would not work as overlaps with
`LUA_REGISTRYINDEX` and below.
* Type mismatch errors now mention if unification failed in covariant or
invariant context, to explain why sometimes derived class can't be
converted to base class or why `T` can't be converted into `T?` and so
on
* Class type indexing is no longer an error in non-strict mode (still an
error in strict mode)
* Fixed cyclic type packs not being displayed in the type
* Added an error when unrelated types are compared with `==`/`~=`
* Fixed false positive errors involving sub-type tests an `never` type
* Fixed miscompilation of multiple assignment statements (Fixes
https://github.com/Roblox/luau/issues/754)
* Type inference stability improvements
* Reduce the stack utilization of type checking.
* Improve the error message that's reported when a delimiting comma is
missing from a table literal. eg
```lua
local t = {
first = 1
second = 2
}```
* https://github.com/Roblox/luau/pull/719
* Improved `Failed to unify type packs` error message to be reported as
`Type pack 'X' could not be converted into 'Y'`
* https://github.com/Roblox/luau/pull/722
* 1% reduction in executed instruction count by removing a check in fast
call dispatch
* Additional fixes to reported error location of OOM errors in VM
* Improve `math.sqrt`, `math.floor` and `math.ceil` performance on
additional compilers and platforms (1-2% geomean improvement including
8-9% on math-cordic)
* All thrown exceptions by Luau analysis are derived from
`Luau::InternalCompilerError`
* When a call site has fewer arguments than required, error now reports
the location of the function name instead of the argument to the
function
* https://github.com/Roblox/luau/pull/724
* Fixed https://github.com/Roblox/luau/issues/725
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
* Support `["prop"]` syntax on class definitions in definition files.
(#704)
* Improve type checking performance for complex overloaded functions
* Fix rare cases of incorrect stack traces for out of memory errors at
runtime
