Some userdata objects may need to support manual destruction in addition
to automatic GC. For example, files, threads, GPU resources and objects
with large external allocations.
With Lua, a finalizer can be _generically_ called by invoking the __gc
metamethod manually, but this is currently not possible with tagged
userdata in Luau because it's not possible to query the destructor
associated with an userdata. While it is possible to workaround this by
duplicating the destructor table locally on client side (*), it's more
convenient to deduplicate the data and get the destructor using the API
instead.
(*) Note: a separate destructor table for each VM may be required if the
VMs use different set of tags.
Implementation notes:
1. I first considered adding a typedef for lua_Destructor but
unfortunately there are two kinds of destructors, one with and one
without the lua_State* argument, so I decided against it at this point.
Maybe it should be added later if the destructor API is unified (by
dropping the Lua state pointer argument?).
2. For some reason the conformance test produced warning "qualifier
applied to function type has no meaning; ignored" on VS2017 (possibly
because the test framework does not like function pointers for some
reason?). I silenced this by pulling out the test expressions from those
CHECKs.
* `table.sort` was improved further. It now guarentees N*log(N) time
complexity in the worst case.
* Fix https://github.com/Roblox/luau/issues/880
We are also working on fixing final bugs and crashes in the new type
solver.
On the CodeGen front we have a few things going on:
* We have a smarter register allocator for the x86 JIT
* We lower more instructions on arm64
* The vector constructor builtin is now translated to IR
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Once again, all of our changes this week are for new type solver and the
JIT.
In the new type solver, we fixed cyclic type alias handling and multiple
stability issues.
In the JIT, our main progress was for arm64, where, after lowering 36%
of instructions, we start seeing first Luau functions executing
natively.
For x64, we performed code cleanup and refactoring to allow for future
optimizations.
All of our changes this week have been focused on the new type solver
and the JIT.
As we march toward feature parity with the old solver, we've tightened
up a bunch of lingering issues with overload resolution, unsealed
tables, and type normalization. We've also fixed a bunch of crashes and
assertion failures in the new solver.
On the JIT front, we've started work on an A64 backend, improved the IR
analysis in a bunch of cases, and implemented assembly generation for
the builtin functions `type()` and `typeof()`.
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fix#817
* Fix#850
* Optimize math.floor/ceil/round with SSE4.1
* Results in a ~7-9% speedup on the math-cordic benchmark.
* Optimized table.sort.
* table.sort is now ~4.1x faster (when not using a predicate) and ~2.1x
faster when using a simple predicate. Performance may improve further in
the future.
* Reorganize the memory ownership of builtin type definitions.
* This is a small initial step toward affording parallel typechecking.
The new type solver is coming along nicely. We are working on fixing
crashes and bugs.
A few major changes to native codegen landed this week:
* Fixed lowering of Luau IR mod instruction when first argument is a
constant
* Added VM register data-flow/capture analysis
* Fixed issues with optimizations in unreachable blocks
---------
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed rare use-after-free in analysis during table unification
A lot of work these past months went into two new Luau components:
* A near full rewrite of the typechecker using a new deferred constraint
resolution system
* Native code generation for AoT/JiT compilation of VM bytecode into x64
(avx)/arm64 instructions
Both of these components are far from finished and we don't provide
documentation on building and using them at this point.
However, curious community members expressed interest in learning about
changes that go into these components each week, so we are now listing
them here in the 'sync' pull request descriptions.
---
New typechecker can be enabled by setting
DebugLuauDeferredConstraintResolution flag to 'true'.
It is considered unstable right now, so try it at your own risk.
Even though it already provides better type inference than the current
one in some cases, our main goal right now is to reach feature parity
with current typechecker.
Features which improve over the capabilities of the current typechecker
are marked as '(NEW)'.
Changes to new typechecker:
* Regular for loop index and parameters are now typechecked
* Invalid type annotations on local variables are ignored to improve
autocomplete
* Fixed missing autocomplete type suggestions for function arguments
* Type reduction is now performed to produce simpler types to be
presented to the user (error messages, custom LSPs)
* Internally, complex types like '((number | string) & ~(false?)) |
string' can be produced, which is just 'string | number' when simplified
* Fixed spots where support for unknown and never types was missing
* (NEW) Length operator '#' is now valid to use on top table type, this
type comes up when doing typeof(x) == "table" guards and isn't available
in current typechecker
---
Changes to native code generation:
* Additional math library fast calls are now lowered to x64: math.ldexp,
math.round, math.frexp, math.modf, math.sign and math.clamp
* For autocomplete, additional information is included in Scope for type
alias name locations and names of imported modules
* Improved autocomplete suggestions in 'for' and 'while' loop headers
* String match functions return types are now optional strings and
numbers because match is not guaranteed at runtime
* Fixed build issue on gcc 11 and up (Fixes
https://github.com/Roblox/luau/issues/806)
`lua_getupvalue()` and `lua_setupvalue()` don't behave as expected when
working with Lua closure whose `Proto` has no debug info. The code
currently uses `sizeupvalues` to do bounds checking of upvalue indices,
but that's the size of the upvalue _names_ array. It will always be `0`
if the `Proto` doesn't have debug info.
This uses `nups` instead, and just returns `""` as the upvalue name if
we don't have one, same as for C closures.
Co-authored-by: Harold Cindy <HaroldCindy@users.noreply.github.com>
* Fix autocompletion of if-then-else expressions
* Fix a potential crash surrounding improper use of `%*` in a string
format specifier
* All Python scripts now invoke Python via `python3` rather than
`python`.
* Improved error handling for string interpolation with too many
arguments.
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
* Fixed unions of `nil` types displaying as `?`
* Internal normalization now handles class types which can make
previously failing (incorrectly) sub-typing checks to succeed
* Reduce the stack utilization of type checking.
* Improve the error message that's reported when a delimiting comma is
missing from a table literal. eg
```lua
local t = {
first = 1
second = 2
}```
* Support `["prop"]` syntax on class definitions in definition files.
(#704)
* Improve type checking performance for complex overloaded functions
* Fix rare cases of incorrect stack traces for out of memory errors at
runtime
* Reoptimized math.min/max/bit32 builtins assuming at least 2 arguments are used (1-2% lift on some benchmarks)
* Type errors that mention function types no longer have redundant parenthesis around return type
* Luau REPL now supports --compile=remarks which displays the source code with optimization remarks embedded as comments
* Builtin calls are slightly faster when called with 1-2 arguments (~1% improvement in some benchmarks)
- Type aliases can no longer override primitive types; attempts to do
that will result in a type error
- Fix misleading type error messages for mismatches in expression list
length during assignment
- Fix incorrect type name display in certain cases
- setmetatable/getmetatable are now ~2x faster
- tools/perfstat.py can be used to display statistics about profiles
captured via --profile switch
To my understanding lua_cleartable does not need GC barriers because
it's only removing elements and not modifying the stack. But I'm not a
GC expert so please correct if I'm wrong.
resolves#672
Co-authored-by: Petri Häkkinen <petrih@rmd.remedy.fi>
Luau currently has the following functions in the C API for dealing with
tables without invoking metamethods:
lua_rawgetfield
lua_rawget
lua_rawgeti
lua_rawset
lua_rawseti
This change adds the missing function lua_rawsetfield for consistency
and because it's more efficient to use it in place of plain lua_rawset
which requires pushing the key and value separately.
Co-authored-by: Petri Häkkinen <petrih@rmd.remedy.fi>
- Remove type definitions of
`utf8.nfcnormalize`/`nfdnormalize`/`graphemes` that aren't supported by
standalone Luau library
- Add `lua_costatus` to retrieve extended thread status (similar to
`coroutine.status`)
- Improve GC sweeping performance (2-10% improvement on allocation-heavy
benchmarks)
- Improve ComparisonPrecedence lint suggestions for three-way comparisons (X < Y < Z)
- Improve type checking stability
- Improve location information for errors when parsing invalid type annotations
- Compiler now generates bytecode version 3 in all configurations
- Improve performance of comparisons against numeric constants on AArch64
- Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative
- Fix an off-by-one error in type error text for incorrect use of string.format
- Reduce stack consumption further during parsing, hopefully eliminating stack overflows during parsing/compilation for good
- Mark interpolated string support as experimental (requires --fflags=LuauInterpolatedStringBaseSupport to enable)
- Simplify garbage collection treatment of upvalues, reducing cache misses during sweeping stage and reducing the cost of upvalue assignment (SETUPVAL); supersedes #643
- Simplify garbage collection treatment of sleeping threads
- Simplify sweeping of alive threads, reducing cache misses during sweeping stage
- Simplify management of string buffers, removing redundant linked list operations
Implements the string interpolation RFC (#165).
Adds the string interpolation as per the RFC.
```lua
local name = "world"
print(`Hello {name}!`) -- Hello world!
```
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com>
- Fix autocomplete not suggesting globals defined after the cursor (fixes#622)
- Improve type checker stability
- Reduce parser C stack consumption which fixes some stack overflow crashes on deeply nested sources
- Improve performance of bit32.extract/replace when width is implied (~3% faster chess)
- Improve performance of bit32.extract when field/width are constants (~10% faster base64)
- Heap dump now annotates thread stacks with local variable/function names
In case a large userdata size is passed to lua_newuserdatadtor it might overflow the size resulting in luaU_newudata actually allocating the object without a memory error. This will then result in overwriting part of the metatable pointer of the userdata.
This PR fixes this issue by checking for the overflow and in such cases pass a size value which will cause a memory error in luaU_newudata.
