Commit graph

173 commits

Author SHA1 Message Date
Vyacheslav Egorov
a9becc9b70 Merge branch 'upstream' into merge 2023-06-09 15:21:03 +03:00
Vyacheslav Egorov
88cd3dda87 Sync to upstream/release/580 2023-06-09 15:20:36 +03:00
Andy Friesen
a3d22decf8 Merge branch 'upstream' into merge 2023-06-02 11:18:16 -07:00
Andy Friesen
f4357400c5 Sync to upstream/release/579 2023-06-02 11:17:31 -07:00
Vyacheslav Egorov
fc9557db1e Merge branch 'upstream' into merge 2023-05-25 23:47:23 +03:00
Vyacheslav Egorov
b8e9d07b20 Sync to upstream/release/578 2023-05-25 23:46:51 +03:00
Andy Friesen
f3a6e37324 Merge branch 'upstream' into merge 2023-05-19 12:03:51 -07:00
Andy Friesen
eb7106016e Sync to upstream/release/577 2023-05-19 11:59:59 -07:00
mundusnine
cf0ba32597
Enable compile-time user configuration for LUA_VECTOR_SIZE (#929)
Add #ifndef guard to enable users to define at compile-time without
having to modify the luau source.
2023-05-16 05:04:49 -07:00
vegorov-rbx
97965c7c0a
Sync to upstream/release/576 (#928)
* `ClassType` can now have an indexer defined on it. This allows custom
types to be used in `t[x]` expressions.
* Fixed search for closest executable breakpoint line. Previously,
breakpoints might have been skipped in `else` blocks at the end of a
function
* Fixed how unification is performed for two optional types `a? <: b?`,
previously it might have unified either 'a' or 'b' with 'nil'. Note that
this fix is not enabled by default yet (see the list in
`ExperimentalFlags.h`)

In the new type solver, a concept of 'Type Families' has been
introduced.
Type families can be thought of as type aliases with custom type
inference/reduction logic included with them.
For example, we can have an `Add<T, U>` type family that will resolve
the type that is the result of adding two values together.
This will help type inference to figure out what 'T' and 'U' might be
when explicit type annotations are not provided.
In this update we don't define any type families, but they will be added
in the near future.
It is also possible for Luau embedders to define their own type families
in the global/environment scope.

Other changes include:
* Fixed scope used to find out which generic types should be included in
the function generic type list
* Fixed a crash after cyclic bound types were created during unification

And in native code generation (jit):
* Use of arm64 target on M1 now requires macOS 13
* Entry into native code has been optimized. This is especially
important for coroutine call/pcall performance as they involve going
through a C call frame
* LOP_LOADK(X) translation into IR has been improved to enable type
tag/constant propagation
* arm64 can use integer immediate values to synthesize floating-point
values
* x64 assembler removes duplicate 64bit numbers from the data section to
save space
* Linux `perf` can now be used to profile native Luau code (when running
with --codegen-perf CLI argument)
2023-05-12 10:50:47 -07:00
Vyacheslav Egorov
3247aabf75 Sync to upstream/release/576 2023-05-12 15:15:01 +03:00
Andy Friesen
8453570658
Sync to upstream/release/575 (#919)
* `Luau.Analyze.CLI` now has experimental support for concurrent type
checking. Use the option `-jN` where `N` is the number of threads to
spawn.
* Improve typechecking performance by ~17% by making the function
`Luau::follow` much more efficient.
* Tighten up the type of `os.date`
* Removed `ParseOptions::allowTypeAnnotations` and
`ParseOptions::supportContinueStatement`

New solver

* Improve the reliability of function overload resolution
* More work toward supporting parallel type checking
* Fix a bug in inference of `==` and `~=` which would erroneously infer
that the operands were `boolean`
* Better error reporting when `for...in` loops are used incorrectly.

CodeGen

* Fix unwind registration when libunwind is used on Linux
* Fixed replaced IR instruction use count
* Convert X64 unwind info generation to standard prologue
* Implement A64 unwind info support for Dwarf2
* Live in/out data for linear blocks is now created
* Add side-exit VM register requirements to the IR dump
* Reuse ConstPropState between block chains 
* Remove redundant base update

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-05-05 14:52:49 -07:00
Andy Friesen
716f63321a Sync to upstream/release/575 2023-05-05 12:57:12 -07:00
vegorov-rbx
4b267aa5c5
Sync to upstream/release/574 (#910)
* Added a limit on how many instructions the Compiler can safely produce
(reported by @TheGreatSageEqualToHeaven)

C++ API Changes:
* With work started on read-only and write-only properties,
`Property::type` member variable has been replaced with `TypeId type()`
and `setType(TypeId)` functions.
* New `LazyType` unwrap callback now has a `void` return type, all
that's required from the callback is to write into `unwrapped` field.

In our work on the new type solver, the following issues were fixed:

* Work has started to support https://github.com/Roblox/luau/pull/77 and
https://github.com/Roblox/luau/pull/79
* Refinements are no longer applied on l-values, removing some
false-positive errors
* Improved overload resolution against expected result type
* `Frontend::prepareModuleScope` now works in the new solver
* Cofinite strings are now comparable

And these are the changes in native code generation (JIT):

* Fixed MIN_NUM and MAX_NUM constant fold when one of the arguments is
NaN
* Added constant folding for number conversions and bit operations
* Value spilling and rematerialization is now supported on arm64
* Improved FASTCALL2K IR generation to support second argument constant
* Added value numbering and load/store propagation optimizations
* Added STORE_VECTOR on arm64, completing the IR lowering on this target
2023-04-28 12:55:13 -07:00
Vyacheslav Egorov
1c2ce0d731 Sync to upstream/release/574 2023-04-28 14:55:55 +03:00
Andy Friesen
d5cdb687e0 Sync to upstream/release/573 2023-04-21 14:41:03 -07:00
vegorov-rbx
d141a5c48d
Sync to upstream/release/572 (#899)
* Fixed exported types not being suggested in autocomplete
* `T...` is now convertible to `...any` (Fixes
https://github.com/Roblox/luau/issues/767)
* Fixed issue with `T?` not being convertible to `T | T` or `T?`
(sometimes when internal pointer identity is different)
* Fixed potential crash in missing table key error suggestion to use a
similar existing key
* `lua_topointer` now returns a pointer for strings

C++ API Changes:
* `prepareModuleScope` callback has moved from TypeChecker to Frontend
* For LSPs, AstQuery functions (and `isWithinComment`) can be used
without full Frontend data

A lot of changes in our two experimental components as well.

In our work on the new type-solver, the following issues were fixed:
* Fixed table union and intersection indexing
* Correct custom type environments are now used
* Fixed issue with values of `free & number` type not accepted in
numeric operations

And these are the changes in native code generation (JIT):
* arm64 lowering is almost complete with support for 99% of IR commands
and all fastcalls
* Fixed x64 assembly encoding for extended byte registers
* More external x64 calls are aware of register allocator
* `math.min`/`math.max` with more than 2 arguments are now lowered to IR
as well
* Fixed correctness issues with `math` library calls with multiple
results in variadic context and with x64 register conflicts
* x64 register allocator learnt to restore values from VM memory instead
of always using stack spills
* x64 exception unwind information now supports multiple functions and
fixes function start offset in Dwarf2 info
2023-04-14 11:06:22 -07:00
Vyacheslav Egorov
5e771b87ae Sync to upstream/release/572 2023-04-14 15:05:27 +03:00
Petri Häkkinen
7345891f6b
Add lua_getuserdatadtor (#870)
Some userdata objects may need to support manual destruction in addition
to automatic GC. For example, files, threads, GPU resources and objects
with large external allocations.

With Lua, a finalizer can be _generically_ called by invoking the __gc
metamethod manually, but this is currently not possible with tagged
userdata in Luau because it's not possible to query the destructor
associated with an userdata. While it is possible to workaround this by
duplicating the destructor table locally on client side (*), it's more
convenient to deduplicate the data and get the destructor using the API
instead.

(*) Note: a separate destructor table for each VM may be required if the
VMs use different set of tags.

Implementation notes:

1. I first considered adding a typedef for lua_Destructor but
unfortunately there are two kinds of destructors, one with and one
without the lua_State* argument, so I decided against it at this point.
Maybe it should be added later if the destructor API is unified (by
dropping the Lua state pointer argument?).

2. For some reason the conformance test produced warning "qualifier
applied to function type has no meaning; ignored" on VS2017 (possibly
because the test framework does not like function pointers for some
reason?). I silenced this by pulling out the test expressions from those
CHECKs.
2023-04-11 12:46:55 -07:00
Andy Friesen
ba67fb275e
Sync to upstream/release/571 (#895)
* `table.sort` was improved further. It now guarentees N*log(N) time
complexity in the worst case.
* Fix https://github.com/Roblox/luau/issues/880

We are also working on fixing final bugs and crashes in the new type
solver.

On the CodeGen front we have a few things going on:
* We have a smarter register allocator for the x86 JIT
* We lower more instructions on arm64
* The vector constructor builtin is now translated to IR

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-04-07 14:01:29 -07:00
Andy Friesen
5309401f49 Sync to upstream/release/571 2023-04-07 12:56:27 -07:00
vegorov-rbx
1212fdacbf
Sync to upstream/release/570 (#885)
Once again, all of our changes this week are for new type solver and the
JIT.

In the new type solver, we fixed cyclic type alias handling and multiple
stability issues.

In the JIT, our main progress was for arm64, where, after lowering 36%
of instructions, we start seeing first Luau functions executing
natively.
For x64, we performed code cleanup and refactoring to allow for future
optimizations.
2023-03-31 11:42:49 -07:00
Vyacheslav Egorov
d1acde36bb Sync to upstream/release/570 2023-03-31 15:21:14 +03:00
Andy Friesen
b4ebad4862
Sync to upstream/release/569 (#878)
All of our changes this week have been focused on the new type solver
and the JIT.

As we march toward feature parity with the old solver, we've tightened
up a bunch of lingering issues with overload resolution, unsealed
tables, and type normalization. We've also fixed a bunch of crashes and
assertion failures in the new solver.

On the JIT front, we've started work on an A64 backend, improved the IR
analysis in a bunch of cases, and implemented assembly generation for
the builtin functions `type()` and `typeof()`.

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-03-24 11:03:04 -07:00
Andy Friesen
81200e13f6 Sync to upstream/release/569 2023-03-24 10:34:14 -07:00
vegorov-rbx
42a2805f85
Sync to upstream/release/568 (#865)
* A small subset of control-flow refinements have been added to
recognize type options that are unreachable after a
conditional/unconditional code block. (Fixes
https://github.com/Roblox/luau/issues/356).

Some examples:
```lua
local function f(x: string?)
    if not x then return end

    -- x is 'string' here
end
```
Throwing calls like `error` or `assert(false)` instead of 'return' are
also recognized.
Existing complex refinements like type/typeof and tagged union checks
are expected to work, among others.

To enable this feature, `LuauTinyControlFlowAnalysis` exclusion has to
be removed from `ExperimentalFlags.h`.
If will become enabled unconditionally in the near future.

* Linter has been integrated into the typechecker analysis so that
type-aware lint warnings can work in any mode
`Frontend::lint` methods were deprecated, `Frontend::check` has to be
used instead with `runLintChecks` option set.
Resulting lint warning are located inside `CheckResult`.

* Fixed large performance drop and increased memory consumption when
array is filled at an offset (Fixes
https://github.com/Roblox/luau/issues/590)
* Part of [Type error suppression
RFC](https://github.com/Roblox/luau/blob/master/rfcs/type-error-suppression.md)
was implemented making subtyping checks with `any` type transitive.

---
In our work on the new type-solver:
* `--!nocheck` mode no longer reports type errors
* New solver will not be used for `--!nonstrict` modules until all
issues with strict mode typechecking are fixed
* Added control-flow aware type refinements mentioned earlier

In native code generation:
* `LOP_NAMECALL` has been translated to IR
* `type` and `typeof` builtin fastcalls have been translated to
IR/assembly
* Additional steps were taken towards arm64 support
2023-03-17 12:20:37 -07:00
Vyacheslav Egorov
e280064f45 Sync to upstream/release/568 2023-03-17 16:59:30 +02:00
Andy Friesen
1fa8311a18
Sync to upstream/release/567 (#860)
* Fix #817 
* Fix #850 
* Optimize math.floor/ceil/round with SSE4.1
    * Results in a ~7-9% speedup on the math-cordic benchmark.
* Optimized table.sort.
* table.sort is now ~4.1x faster (when not using a predicate) and ~2.1x
faster when using a simple predicate. Performance may improve further in
the future.
* Reorganize the memory ownership of builtin type definitions.
* This is a small initial step toward affording parallel typechecking.

The new type solver is coming along nicely. We are working on fixing
crashes and bugs.

A few major changes to native codegen landed this week:
* Fixed lowering of Luau IR mod instruction when first argument is a
constant
* Added VM register data-flow/capture analysis
* Fixed issues with optimizations in unreachable blocks

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-03-10 12:21:07 -08:00
Andy Friesen
4653484913 Sync to upstream/release/567 2023-03-10 11:20:04 -08:00
niansa/tuxifan
78798d4641
Fixed lerp() duplicate when compiling for C++20 (#851) 2023-03-06 04:42:59 -08:00
vegorov-rbx
140e5a1495
Sync to upstream/release/566 (#853)
* Fixed incorrect lexeme generated for string parts in the middle of an
interpolated string (Fixes https://github.com/Roblox/luau/issues/744)
* DeprecatedApi lint can report some issues without type inference
information
* Fixed performance of autocomplete requests when suggestions have large
intersection types (Solves
https://github.com/Roblox/luau/discussions/847)
* Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC:
Deprecate
table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md))
* With -O2 optimization level, we now optimize builtin calls based on
known argument/return count.
Note that this change can be observable if `getfenv/setfenv` is used to
substitute a builtin, especially if arity is different.
Fastcall heavy tests show a 1-2% improvement.
* Luau can now be built with clang-cl (Fixes
https://github.com/Roblox/luau/issues/736)

We also made many improvements to our experimental components.

For our new type solver:
* Overhauled data flow analysis system, fixed issues with 'repeat'
loops, global variables and type annotations
* Type refinements now work on generic table indexing with a string
literal
* Type refinements will properly track potentially 'nil' values (like
t[x] for a missing key) and their further refinements
* Internal top table type is now isomorphic to `{}` which fixes issues
when `typeof(v) == 'table'` type refinement is handled
* References to non-existent types in type annotations no longer resolve
to 'error' type like in old solver
* Improved handling of class unions in property access expressions
* Fixed default type packs
* Unsealed tables can now have metatables
* Restored expected types for function arguments

And for native code generation:
* Added min and max IR instructions mapping to vminsd/vmaxsd on x64
* We now speculatively extract direct execution fast-paths based on
expected types of expressions which provides better optimization
opportunities inside a single basic block
* Translated existing math fastcalls to IR form to improve tag guard
removal and constant propagation
2023-03-03 12:21:14 -08:00
Vyacheslav Egorov
9a281f0492 Sync to upstream/release/566 2023-03-03 15:45:38 +02:00
vegorov-rbx
62483d40f0
Sync to upstream/release/562 (#828)
* Fixed rare use-after-free in analysis during table unification

A lot of work these past months went into two new Luau components:
* A near full rewrite of the typechecker using a new deferred constraint
resolution system
* Native code generation for AoT/JiT compilation of VM bytecode into x64
(avx)/arm64 instructions

Both of these components are far from finished and we don't provide
documentation on building and using them at this point.
However, curious community members expressed interest in learning about
changes that go into these components each week, so we are now listing
them here in the 'sync' pull request descriptions.

---
New typechecker can be enabled by setting
DebugLuauDeferredConstraintResolution flag to 'true'.
It is considered unstable right now, so try it at your own risk.
Even though it already provides better type inference than the current
one in some cases, our main goal right now is to reach feature parity
with current typechecker.
Features which improve over the capabilities of the current typechecker
are marked as '(NEW)'.

Changes to new typechecker:
* Regular for loop index and parameters are now typechecked
* Invalid type annotations on local variables are ignored to improve
autocomplete
* Fixed missing autocomplete type suggestions for function arguments
* Type reduction is now performed to produce simpler types to be
presented to the user (error messages, custom LSPs)
* Internally, complex types like '((number | string) & ~(false?)) |
string' can be produced, which is just 'string | number' when simplified
* Fixed spots where support for unknown and never types was missing
* (NEW) Length operator '#' is now valid to use on top table type, this
type comes up when doing typeof(x) == "table" guards and isn't available
in current typechecker

---
Changes to native code generation:
* Additional math library fast calls are now lowered to x64: math.ldexp,
math.round, math.frexp, math.modf, math.sign and math.clamp
2023-02-03 11:26:13 -08:00
Vyacheslav Egorov
dba2936823 Sync to upstream/release/562 2023-02-03 14:34:12 +02:00
Andy Friesen
f763f4c948
Sync to upstream/release/561 (#820)
* Fix a potential debugger crash by adding checks for invalid stack
index values

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-01-27 14:28:31 -08:00
Andy Friesen
53d03f94f7 Sync to upstream/release/561 2023-01-27 13:28:45 -08:00
Vyacheslav Egorov
eec289ad1b Sync to upstream/release/560 2023-01-20 14:02:39 +02:00
Harold Cindy
729bc44729
Fix lua_*upvalue() when upvalue names aren't in debug info (#787)
`lua_getupvalue()` and `lua_setupvalue()` don't behave as expected when
working with Lua closure whose `Proto` has no debug info. The code
currently uses `sizeupvalues` to do bounds checking of upvalue indices,
but that's the size of the upvalue _names_ array. It will always be `0`
if the `Proto` doesn't have debug info.

This uses `nups` instead, and just returns `""` as the upvalue name if
we don't have one, same as for C closures.

Co-authored-by: Harold Cindy <HaroldCindy@users.noreply.github.com>
2023-01-18 06:00:13 -08:00
Andy Friesen
a5c6a38b10
Sync to upstream/release/559 (#804)
* Fix autocompletion of if-then-else expressions
* Fix a potential crash surrounding improper use of `%*` in a string
format specifier
* All Python scripts now invoke Python via `python3` rather than
`python`.
* Improved error handling for string interpolation with too many
arguments.

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-01-13 14:10:01 -08:00
Andy Friesen
96c1cafff2 Sync to upstream/release/559 2023-01-13 12:36:28 -08:00
Vyacheslav Egorov
36f5009026 Sync to upstream/release/558 2023-01-06 18:07:19 +02:00
Alex Orlenko
e9d4ee7a33
Make pseudo-indices relative to LUAI_MAXCSTACK (#766)
This is useful in particular if redefine `LUAI_MAXCSTACK` to a higher
value than the current one (8000).
Ie. passing `-D LUAI_MAXCSTACK=1000000` would not work as overlaps with
`LUA_REGISTRYINDEX` and below.
2022-12-06 11:20:24 -08:00
vegorov-rbx
59ae47db43
Sync to upstream/release/555 (#768)
* Type mismatch errors now mention if unification failed in covariant or
invariant context, to explain why sometimes derived class can't be
converted to base class or why `T` can't be converted into `T?` and so
on
* Class type indexing is no longer an error in non-strict mode (still an
error in strict mode)
* Fixed cyclic type packs not being displayed in the type
* Added an error when unrelated types are compared with `==`/`~=`
* Fixed false positive errors involving sub-type tests an `never` type
* Fixed miscompilation of multiple assignment statements (Fixes
https://github.com/Roblox/luau/issues/754)
* Type inference stability improvements
2022-12-02 10:09:59 -08:00
Vyacheslav Egorov
fc459699da Sync to upstream/release/555 2022-12-02 12:46:05 +02:00
Andy Friesen
95d9c6d194
Sync to upstream/release/553 (#751)
* Autocomplete support for interpolated strings.
* Improved parse errors in various situations involving interpolated
strings.
2022-11-18 11:47:21 -08:00
Andy Friesen
f52169509c Sync to upstream/release/554 2022-11-18 10:45:14 -08:00
Andy Friesen
c33700e473
Sync to upstream/release/552 (#735)
* Reduce the stack utilization of type checking.
* Improve the error message that's reported when a delimiting comma is
missing from a table literal. eg
```lua
local t = {
    first = 1
    second = 2
}```
2022-11-04 10:33:22 -07:00
Andy Friesen
e3fdab3082 Sync to upstream/release/552 2022-11-04 10:02:37 -07:00
vegorov-rbx
a6b5051edc
Sync to upstream/release/551 (#727)
* https://github.com/Roblox/luau/pull/719
* Improved `Failed to unify type packs` error message to be reported as
`Type pack 'X' could not be converted into 'Y'`
* https://github.com/Roblox/luau/pull/722
* 1% reduction in executed instruction count by removing a check in fast
call dispatch
* Additional fixes to reported error location of OOM errors in VM
* Improve `math.sqrt`, `math.floor` and `math.ceil` performance on
additional compilers and platforms (1-2% geomean improvement including
8-9% on math-cordic)
* All thrown exceptions by Luau analysis are derived from
`Luau::InternalCompilerError`
* When a call site has fewer arguments than required, error now reports
the location of the function name instead of the argument to the
function
* https://github.com/Roblox/luau/pull/724
* Fixed https://github.com/Roblox/luau/issues/725

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
2022-10-28 03:37:29 -07:00
Vyacheslav Egorov
99c0db3b08 Sync to upstream/release/551 2022-10-28 01:22:49 +03:00