## What's Changed
- Improve POSIX compliance in `CLI/FileUtils.cpp` by @SamuraiCrow #1064
- `AstStat*::hasEnd` is deprecated; use `AstStatBlock::hasEnd` instead
- Added a lint for common misuses of the `#` operator
- Luau now issues deprecated diagnostics for some uses of `getfenv` and
`setfenv`
- Fixed a case where we included a trailing space in some error
stringifications
### Compiler
- Do not do further analysis in O2 on self functions
- Improve detection of invalid repeat..until expressions vs continue
### New Type Solver
- We now cache subtype test results to improve performance
- Improved operator inference mechanics (aka type families)
- Further work towards type states
- Work towards [new non-strict
mode](https://github.com/Roblox/luau/blob/master/rfcs/new-nonstrict.md)
continues
### Native Codegen
- Instruction last use locations should follow the order in which blocks
are lowered
- Add a bonus assertion to IrLoweringA64::tempAddr
---
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Adds an option `compositeTypesSingleLineLimit` to control when
unions/intersections are broken up onto multiple lines when printed.
Simple unions/intersections will remain on one line. Default value is
set to `5`.
I took the liberty to always expand overloaded function intersections on
multiple lines, as I believe they are best viewed that way, even if
theres only 2 functions.
Closes#963
# New Type Solver
- Implement bidirectional type inference for higher order functions so
that we can provide a more precise type improving the autocomplete's
human factors.
- We seal all tables, so we changed the stringification to make it a
little lighter on users.
- Fixed a case of array-out-of-bound access.
- Type families no longer depends on `TxnLog` and `Unifier`.
- Type refinements now waits until the free types are sufficiently
solved.
# Native Code Generation
- Remove cached slot lookup for `executeSETTABLEKS` function because it
is a fallback in the event of a cache miss, making the cached slot
lookup redundant.
- Optimized repeated array lookups, e.g. `a[3]` in `a[3] = a[3] / 2` is
done once.
# Misc
- On some platforms, it is necessary to use `gmtime_s` with the
arguments reversed to get the current time. You can now define
`DOCTEST_CONFIG_USE_GMTIME_S` to build and run unit tests on those
platforms.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
- Cleaned up `FFlag::FixFindBindingAtFunctionName`,
`FFlag::LuauNormalizeBlockedTypes`, `FFlag::LuauPCallDebuggerFix`
- Added support for break and continue into control flow analysis
- The old type unification engine will now report a more fine-grained
error at times, indicating that type normalization in particular failed
# New Type Solver
- Refactor of Unifier2, the new unification implementation for Luau
- Completed MVP of new unification implementation
- Dramatically simplified overload selection logic
- Type family reduction can now apply sooner to free types that have
been solved
- Subtyping now supports table indexers
- Generalization now replaces bad generics with unknown
# Native Code Generation
- Reduce stack spills caused by FINDUPVAL and STORE_TAG
- Improve Generate SHL/SHR/SAR/rotates with immediate operands in X64
- Removed redundant case re-check in table lookup fallback
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
- Updated Roblox copyright to 2023
- Floor division operator `//` (implements #832)
- Autocomplete now shows `end` within `do` blocks
- Restore BraceType when using `Lexer::lookahead` (fixes#1019)
# New typechecker
- Subtyping tests between metatables and tables
- Subtyping tests between string singletons and tables
- Subtyping tests for class types
# Native codegen
- Fixed macOS test failure (wrong spill restore offset)
- Fixed clobbering of non-volatile xmm registers on Windows
- Fixed wrong storage location of SSA reg spills
- Implemented A64 support for add/sub extended
- Eliminated zextReg from A64 lowering
- Remove identical table slot lookups
- Propagate values from predecessor into the linear block
- Disabled reuse slot optimization
- Keep `LuaNode::val` check for nil when optimizing `CHECK_SLOT_MATCH`
- Implemented IR translation of `table.insert` builtin
- Fixed mmap error handling on macOS/Linux
# Tooling
- Used `|` as a column separator instead of `+` in `bench.py`
- Added a `table.sort` micro-benchmark
- Switched `libprotobuf-mutator` to a less problematic version
* Better indentation in multi-line type mismatch error messages
* Error message clone can no longer cause a stack overflow (when
typechecking with retainFullTypeGraphs set to false); fixes
https://github.com/Roblox/luau/issues/975
* `string.format` with %s is now ~2x faster on strings smaller than 100
characters
Native code generation:
* All VM side exits will block return to the native execution of the
current function to preserve correctness
* Fixed executable page allocation on Apple platforms when using
hardened runtime
* Added statistics for code generation (no. of functions compiler,
memory used for different areas)
* Fixed issue with function entry type checks performed more that once
in some functions
Type checker/autocomplete:
* `Luau::autocomplete` no longer performs typechecking internally, make
sure to run `Frontend::check` before performing autocomplete requests
* Autocomplete string suggestions without "" are now only suggested
inside the ""
* Autocomplete suggestions now include `function (anonymous autofilled)`
key with a full suggestion for the function expression (with arguments
included) stored in `AutocompleteEntry::insertText`
* `AutocompleteEntry::indexedWithSelf` is provided for function call
suggestions made with `:`
* Cyclic modules now see each other type exports as `any` to prevent
memory use-after-free (similar to module return type)
Runtime:
* Updated inline/loop unroll cost model to better handle assignments
(Fixes https://github.com/Roblox/luau/issues/978)
* `math.noise` speed was improved by ~30%
* `table.concat` speed was improved by ~5-7%
* `tonumber` and `tostring` now have fastcall paths that execute ~1.5x
and ~2.5x faster respectively (fixes#777)
* Fixed crash in `luaL_typename` when index refers to a non-existing
value
* Fixed potential out of memory scenario when using `string.sub` or
`string.char` in a loop
* Fixed behavior of some fastcall builtins when called without arguments
under -O2 to match original functions
* Support for native code execution in VM is now enabled by default
(note: native code still has to be generated explicitly)
* `Codegen::compile` now accepts `CodeGen_OnlyNativeModules` flag. When
set, only modules that have a `--!native` hot-comment at the top will be
compiled to native code
In our new typechecker:
* Generic type packs are no longer considered to be variadic during
unification
* Timeout and cancellation now works in new solver
* Fixed false positive errors around 'table' and 'function' type
refinements
* Table literals now use covariant unification rules. This is sound
since literal has no type specified and has no aliases
* Fixed issues with blocked types escaping the constraint solver
* Fixed more places where error messages that should've been suppressed
were still reported
* Fixed errors when iterating over a top table type
In our native code generation (jit):
* 'DebugLuauAbortingChecks' flag is now supported on A64
* LOP_NEWCLOSURE has been translated to IR
Lots of things going on this week:
* Fix a crash that could occur in the presence of a cyclic union. We
shouldn't be creating cyclic unions, but we shouldn't be crashing when
they arise either.
* Minor cleanup of `luau_precall`
* Internal change to make L->top handling slightly more uniform
* Optimize SETGLOBAL & GETGLOBAL fallback C functions.
* https://github.com/Roblox/luau/pull/929
* The syntax to the `luau-reduce` commandline tool has changed. It now
accepts a script, a command to execute, and an error to search for. It
no longer automatically passes the script to the command which makes it
a lot more flexible. Also be warned that it edits the script it is
passed **in place**. Do not point it at something that is not in source
control!
New solver
* Switch to a greedier but more fallible algorithm for simplifying union
and intersection types that are created as part of refinement
calculation. This has much better and more predictable performance.
* Fix a constraint cycle in recursive function calls.
* Much improved inference of binary addition. Functions like `function
add(x, y) return x + y end` can now be inferred without annotations. We
also accurately typecheck calls to functions like this.
* Many small bugfixes surrounding things like table indexers
* Add support for indexers on class types. This was previously added to
the old solver; we now add it to the new one for feature parity.
JIT
* https://github.com/Roblox/luau/pull/931
* Fuse key.value and key.tt loads for CEHCK_SLOT_MATCH in A64
* Implement remaining aliases of BFM for A64
* Implement new callinfo flag for A64
* Add instruction simplification for int->num->int conversion chains
* Don't even load execdata for X64 calls
* Treat opcode fallbacks the same as manually written fallbacks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Added a limit on how many instructions the Compiler can safely produce
(reported by @TheGreatSageEqualToHeaven)
C++ API Changes:
* With work started on read-only and write-only properties,
`Property::type` member variable has been replaced with `TypeId type()`
and `setType(TypeId)` functions.
* New `LazyType` unwrap callback now has a `void` return type, all
that's required from the callback is to write into `unwrapped` field.
In our work on the new type solver, the following issues were fixed:
* Work has started to support https://github.com/Roblox/luau/pull/77 and
https://github.com/Roblox/luau/pull/79
* Refinements are no longer applied on l-values, removing some
false-positive errors
* Improved overload resolution against expected result type
* `Frontend::prepareModuleScope` now works in the new solver
* Cofinite strings are now comparable
And these are the changes in native code generation (JIT):
* Fixed MIN_NUM and MAX_NUM constant fold when one of the arguments is
NaN
* Added constant folding for number conversions and bit operations
* Value spilling and rematerialization is now supported on arm64
* Improved FASTCALL2K IR generation to support second argument constant
* Added value numbering and load/store propagation optimizations
* Added STORE_VECTOR on arm64, completing the IR lowering on this target
* Fix#817
* Fix#850
* Optimize math.floor/ceil/round with SSE4.1
* Results in a ~7-9% speedup on the math-cordic benchmark.
* Optimized table.sort.
* table.sort is now ~4.1x faster (when not using a predicate) and ~2.1x
faster when using a simple predicate. Performance may improve further in
the future.
* Reorganize the memory ownership of builtin type definitions.
* This is a small initial step toward affording parallel typechecking.
The new type solver is coming along nicely. We are working on fixing
crashes and bugs.
A few major changes to native codegen landed this week:
* Fixed lowering of Luau IR mod instruction when first argument is a
constant
* Added VM register data-flow/capture analysis
* Fixed issues with optimizations in unreachable blocks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
We've made a few small changes to reduce the amount of stack we use when
typechecking nested method calls (eg `foo:bar():baz():quux()`).
We've also fixed a small bytecode compiler issue that caused us to emit
redundant jump instructions in code that conditionally uses `break` or
`continue`.
On the new solver, we've switched to a new, better way to handle
augmentations to unsealed tables. We've also made some substantial
improvements to type inference and error reporting on function calls.
These things should both be on par with the old solver now.
The main improvements to the native code generator have been elimination
of some redundant type tag checks. Also, we are starting to inline
particular fastcalls directly to IR.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed rare use-after-free in analysis during table unification
A lot of work these past months went into two new Luau components:
* A near full rewrite of the typechecker using a new deferred constraint
resolution system
* Native code generation for AoT/JiT compilation of VM bytecode into x64
(avx)/arm64 instructions
Both of these components are far from finished and we don't provide
documentation on building and using them at this point.
However, curious community members expressed interest in learning about
changes that go into these components each week, so we are now listing
them here in the 'sync' pull request descriptions.
---
New typechecker can be enabled by setting
DebugLuauDeferredConstraintResolution flag to 'true'.
It is considered unstable right now, so try it at your own risk.
Even though it already provides better type inference than the current
one in some cases, our main goal right now is to reach feature parity
with current typechecker.
Features which improve over the capabilities of the current typechecker
are marked as '(NEW)'.
Changes to new typechecker:
* Regular for loop index and parameters are now typechecked
* Invalid type annotations on local variables are ignored to improve
autocomplete
* Fixed missing autocomplete type suggestions for function arguments
* Type reduction is now performed to produce simpler types to be
presented to the user (error messages, custom LSPs)
* Internally, complex types like '((number | string) & ~(false?)) |
string' can be produced, which is just 'string | number' when simplified
* Fixed spots where support for unknown and never types was missing
* (NEW) Length operator '#' is now valid to use on top table type, this
type comes up when doing typeof(x) == "table" guards and isn't available
in current typechecker
---
Changes to native code generation:
* Additional math library fast calls are now lowered to x64: math.ldexp,
math.round, math.frexp, math.modf, math.sign and math.clamp
* For autocomplete, additional information is included in Scope for type
alias name locations and names of imported modules
* Improved autocomplete suggestions in 'for' and 'while' loop headers
* String match functions return types are now optional strings and
numbers because match is not guaranteed at runtime
* Fixed build issue on gcc 11 and up (Fixes
https://github.com/Roblox/luau/issues/806)
* Fix autocompletion of if-then-else expressions
* Fix a potential crash surrounding improper use of `%*` in a string
format specifier
* All Python scripts now invoke Python via `python3` rather than
`python`.
* Improved error handling for string interpolation with too many
arguments.
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed unions of `nil` types displaying as `?`
* Internal normalization now handles class types which can make
previously failing (incorrectly) sub-typing checks to succeed
* The AST JSON encoder will now stringify infinity and NaN according to the JSON5 spec using the tokens `Infinity`, `-Infinity`, and `NaN`.
* Improve autocompletion of table keys if the type of that key is a union of string singletons.
* Type mismatch errors now mention if unification failed in covariant or
invariant context, to explain why sometimes derived class can't be
converted to base class or why `T` can't be converted into `T?` and so
on
* Class type indexing is no longer an error in non-strict mode (still an
error in strict mode)
* Fixed cyclic type packs not being displayed in the type
* Added an error when unrelated types are compared with `==`/`~=`
* Fixed false positive errors involving sub-type tests an `never` type
* Fixed miscompilation of multiple assignment statements (Fixes
https://github.com/Roblox/luau/issues/754)
* Type inference stability improvements
* Type inference of `a and b` and `a or b` has been improved (Fixes
https://github.com/Roblox/luau/issues/730)
* Improved error message when `for ... in x` loop iterates over a value
that could be 'nil'
* Return type of `next` not includes 'nil' (Fixes
https://github.com/Roblox/luau/issues/706)
* Improved type inference of `string` type
* Luau library table type names are now reported as `typeof(string)`/etc
instead of just `string` which was misleading
* Added parsing error when optional type alias type parameter wasn't
provided after `=` token
* Improved tagged union type refinement in conditional expressions, type
in `else` branch should no longer include previously handled union
options
* Reduce the stack utilization of type checking.
* Improve the error message that's reported when a delimiting comma is
missing from a table literal. eg
```lua
local t = {
first = 1
second = 2
}```
* https://github.com/Roblox/luau/pull/719
* Improved `Failed to unify type packs` error message to be reported as
`Type pack 'X' could not be converted into 'Y'`
* https://github.com/Roblox/luau/pull/722
* 1% reduction in executed instruction count by removing a check in fast
call dispatch
* Additional fixes to reported error location of OOM errors in VM
* Improve `math.sqrt`, `math.floor` and `math.ceil` performance on
additional compilers and platforms (1-2% geomean improvement including
8-9% on math-cordic)
* All thrown exceptions by Luau analysis are derived from
`Luau::InternalCompilerError`
* When a call site has fewer arguments than required, error now reports
the location of the function name instead of the argument to the
function
* https://github.com/Roblox/luau/pull/724
* Fixed https://github.com/Roblox/luau/issues/725
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
* Reoptimized math.min/max/bit32 builtins assuming at least 2 arguments are used (1-2% lift on some benchmarks)
* Type errors that mention function types no longer have redundant parenthesis around return type
* Luau REPL now supports --compile=remarks which displays the source code with optimization remarks embedded as comments
* Builtin calls are slightly faster when called with 1-2 arguments (~1% improvement in some benchmarks)
- Type aliases can no longer override primitive types; attempts to do
that will result in a type error
- Fix misleading type error messages for mismatches in expression list
length during assignment
- Fix incorrect type name display in certain cases
- setmetatable/getmetatable are now ~2x faster
- tools/perfstat.py can be used to display statistics about profiles
captured via --profile switch
- Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative
- Fix an off-by-one error in type error text for incorrect use of string.format
- Reduce stack consumption further during parsing, hopefully eliminating stack overflows during parsing/compilation for good
- Mark interpolated string support as experimental (requires --fflags=LuauInterpolatedStringBaseSupport to enable)
- Simplify garbage collection treatment of upvalues, reducing cache misses during sweeping stage and reducing the cost of upvalue assignment (SETUPVAL); supersedes #643
- Simplify garbage collection treatment of sleeping threads
- Simplify sweeping of alive threads, reducing cache misses during sweeping stage
- Simplify management of string buffers, removing redundant linked list operations
* Run clang-format
* Contains a preliminary implementation of deferred constraint resolution
* Reduce stack usage by some recursive functions
* Fix a bug when smartCloning a BoundTypeVar
* Remove some GC related flags from VM
* Respect useLineBreaks for union/intersect toString
* Apply suggestions from code review
Co-authored-by: Andy Friesen <andy.friesen@gmail.com>
Co-authored-by: Andy Friesen <andy.friesen@gmail.com>
Adds an option to hide the `self: type` argument as the first argument in the string representation of a named function type var if the ftv hasSelf.
Also added in a test for the original output (i.e., if the option was disabled)
I didn't apply this option in the normal `Luau::toString()` function, just the `Luau::toStringNamedFunction()` one (for my usecase, that is enough + I felt like a named function would include the method colon `:` to signify self). If this is unintuitive, I can also add it to the general `Luau::toString()` function.
- Improve refinement support for unions, in particular it's now possible to implement tagged unions as a union of tables where individual branches use a string literal type for one of the fields.
- Fix `string.split` type information
- Optimize `select(_, ...)` to run in constant time (~2.7x faster on VariadicSelect benchmark)
- Improve debug line information for multi-line assignments
- Improve compilation of table literals when table keys are constant expressions/variables
- Use forward GC barrier for `setmetatable` which slightly accelerates GC progress
- Improve error recovery during type checking
- Initial (not fully complete) implementation for singleton types (RFC RFC: Singleton types #37)
- Implement a C-friendly interface for compiler (luacode.h)
- Remove C++ features from lua.h (removed default arguments from luau_load and lua_pushcfunction)
- Fix lua_breakpoint behavior when enabled=false
- Implement coroutine.close (RFC RFC: coroutine.close #88)
Note, this introduces small breaking changes in lua.h:
- luau_load env argument is now required, pass an extra 0
- lua_pushcfunction now must be called with 3 arguments; if you were calling it with 2 arguments, pass an extra NULL; if you were calling it with 4, use lua_pushcclosure.
These changes are necessary to make sure lua.h can be used from pure C - the future release will make it possible by adding an option to luaconf.h to change function name mangling to be C-compatible. We don't anticipate breaking the FFI interface in the future, but this change was necessary to restore C compatibility.
Closes#121Fixes#213
- Type mismatch errors now show detailed information for compound types, highlighting the mismatching component
- Fix string.pack bug on ARM when packing negative numbers using unsigned formats
- Implement bit32.countlz/countrz (RFC RFC: bit32.countlz/countrz #89)
- Minor compiler throughput optimization (~2% faster compilation)
- Improve transpiler behavior for edge cases and better test coverage (not exposed through CLI at the moment)
- Improve error recovery when parsing invalid assignments
- Build fixes for fuzzing targets
Changes:
- Support for time tracing for analysis/compiler (not currently exposed
through CLI)
- Support for type pack arguments in type aliases (#83)
- Basic support for require(path) in luau-analyze
- Add a lint warning for table.move with 0 index as part of
TableOperation lint
- Remove last STL dependency from Luau.VM
- Minor VS2022 performance tuning
Co-authored-by: Rodactor <rodactor@roblox.com>
