Commit graph

847 commits

Author SHA1 Message Date
Vyacheslav Egorov
ef2e46e25e Merge branch 'master' into merge 2023-06-09 15:20:56 +03:00
Arseny Kapoulkine
febebde72a
Update performance.md (#948)
We haven't updated this document in a while and yet there was a fair
amount of performance work, some of which can be documented here.

Note that this is not fully comprehensive in that it excludes a lot of
internal tuning that is difficult to describe other than with "we made
things faster".
2023-06-06 15:11:29 -07:00
Ben Mactavsin
e78897229a
Fix website demo's string highlighting behaviour (#942)
Fixes #935:
* String literals that include `\z` escape sequence followed by newline
characters are now correctly highlighted.
* Unescaped backslash (`\`) character at the end of the line no longer
acts like the `\z` escape sequence inside string literals when
highlighting.
2023-06-06 11:22:31 -07:00
Ben Mactavsin
4f82a77396
Update lint.md to specify language on fenced code blocks which have none (#946)
Specified language on the last two fenced code blocks in the page as
they were inconsistent with the others on the same page.
2023-06-06 08:57:36 -07:00
Andy Friesen
63679f7288
Sync to upstream/release/579 (#943)
A pretty small changelist this week:

* When type inference fails to find any matching overload for a
function, we were declining to commit any changes to the type graph at
all. This was resulting in confusing type errors in certain cases. Now,
when a matching overload cannot be found, we always commit to the first
overload we tried.

JIT

* Fix missing variadic register invalidation in FALLBACK_GETVARARGS
* Add a missing null pointer check for the result of luaT_gettm

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-06-02 12:52:15 -07:00
Andy Friesen
a3d22decf8 Merge branch 'upstream' into merge 2023-06-02 11:18:16 -07:00
Andy Friesen
349d6f4342 Merge branch 'master' into merge 2023-06-02 11:18:12 -07:00
Andy Friesen
f4357400c5 Sync to upstream/release/579 2023-06-02 11:17:31 -07:00
vegorov-rbx
271c509046
Sync to upstream/release/578 (#939)
* Fixed gcc warning about uninitialized `std::optional`
* Fixed inlining of functions when they are used to compute their own
arguments

In the new type solver:
* Type families that are not part of a function signature cannot be
resolved at instantiation time and will now produce an error. This will
be relaxed in the future when we get constraint clauses on function
signatures (internally)
* `never` type is now comparable
* Improved typechecking of `for..in` statements
* Fixed checks for number type in `Add` type family
* Performance was improved, with particularly large gains on large
projects

And in native code generation (jit):
* We eliminated the call instruction overhead when native code support
is enabled in the VM
* Small optimizations to arm64 lowering
* Reworked LOP_GETIMPORT handling to reduce assembly code size
* Fixed non-deterministic binary output
* Fixed bad code generation caused by incorrect SSA to VM register links
invalidation

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
2023-05-25 14:36:34 -07:00
Vyacheslav Egorov
fc9557db1e Merge branch 'upstream' into merge 2023-05-25 23:47:23 +03:00
Vyacheslav Egorov
f97a897ccd Merge branch 'master' into merge 2023-05-25 23:47:17 +03:00
Vyacheslav Egorov
b8e9d07b20 Sync to upstream/release/578 2023-05-25 23:46:51 +03:00
Andy Friesen
721f6e10fb
Sync to upstream/release/577 (#934)
Lots of things going on this week:

* Fix a crash that could occur in the presence of a cyclic union. We
shouldn't be creating cyclic unions, but we shouldn't be crashing when
they arise either.
* Minor cleanup of `luau_precall`
* Internal change to make L->top handling slightly more uniform
* Optimize SETGLOBAL & GETGLOBAL fallback C functions.
* https://github.com/Roblox/luau/pull/929
* The syntax to the `luau-reduce` commandline tool has changed. It now
accepts a script, a command to execute, and an error to search for. It
no longer automatically passes the script to the command which makes it
a lot more flexible. Also be warned that it edits the script it is
passed **in place**. Do not point it at something that is not in source
control!

New solver

* Switch to a greedier but more fallible algorithm for simplifying union
and intersection types that are created as part of refinement
calculation. This has much better and more predictable performance.
* Fix a constraint cycle in recursive function calls.
* Much improved inference of binary addition. Functions like `function
add(x, y) return x + y end` can now be inferred without annotations. We
also accurately typecheck calls to functions like this.
* Many small bugfixes surrounding things like table indexers
* Add support for indexers on class types. This was previously added to
the old solver; we now add it to the new one for feature parity.

JIT

* https://github.com/Roblox/luau/pull/931
* Fuse key.value and key.tt loads for CEHCK_SLOT_MATCH in A64
* Implement remaining aliases of BFM for A64
* Implement new callinfo flag for A64
* Add instruction simplification for int->num->int conversion chains
* Don't even load execdata for X64 calls
* Treat opcode fallbacks the same as manually written fallbacks

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-05-19 12:37:30 -07:00
Andy Friesen
123496b29c gcc fix. 2023-05-19 12:11:10 -07:00
Andy Friesen
f3a6e37324 Merge branch 'upstream' into merge 2023-05-19 12:03:51 -07:00
Andy Friesen
4fb7594eaa Merge branch 'master' into merge 2023-05-19 12:00:39 -07:00
Andy Friesen
eb7106016e Sync to upstream/release/577 2023-05-19 11:59:59 -07:00
Alex Orlenko
da0458bf6e
Add CodeGen C API (#931)
I'd like to experiment with the codegen feature (currently experimental)
and need a public C API for this.

This PR addresses this.
2023-05-18 04:03:29 -07:00
mundusnine
cf0ba32597
Enable compile-time user configuration for LUA_VECTOR_SIZE (#929)
Add #ifndef guard to enable users to define at compile-time without
having to modify the luau source.
2023-05-16 05:04:49 -07:00
vegorov-rbx
97965c7c0a
Sync to upstream/release/576 (#928)
* `ClassType` can now have an indexer defined on it. This allows custom
types to be used in `t[x]` expressions.
* Fixed search for closest executable breakpoint line. Previously,
breakpoints might have been skipped in `else` blocks at the end of a
function
* Fixed how unification is performed for two optional types `a? <: b?`,
previously it might have unified either 'a' or 'b' with 'nil'. Note that
this fix is not enabled by default yet (see the list in
`ExperimentalFlags.h`)

In the new type solver, a concept of 'Type Families' has been
introduced.
Type families can be thought of as type aliases with custom type
inference/reduction logic included with them.
For example, we can have an `Add<T, U>` type family that will resolve
the type that is the result of adding two values together.
This will help type inference to figure out what 'T' and 'U' might be
when explicit type annotations are not provided.
In this update we don't define any type families, but they will be added
in the near future.
It is also possible for Luau embedders to define their own type families
in the global/environment scope.

Other changes include:
* Fixed scope used to find out which generic types should be included in
the function generic type list
* Fixed a crash after cyclic bound types were created during unification

And in native code generation (jit):
* Use of arm64 target on M1 now requires macOS 13
* Entry into native code has been optimized. This is especially
important for coroutine call/pcall performance as they involve going
through a C call frame
* LOP_LOADK(X) translation into IR has been improved to enable type
tag/constant propagation
* arm64 can use integer immediate values to synthesize floating-point
values
* x64 assembler removes duplicate 64bit numbers from the data section to
save space
* Linux `perf` can now be used to profile native Luau code (when running
with --codegen-perf CLI argument)
2023-05-12 10:50:47 -07:00
Vyacheslav Egorov
59c2698970 Merge branch 'upstream' into merge 2023-05-12 15:30:03 +03:00
Vyacheslav Egorov
38ab2e5f71 Merge branch 'master' into merge 2023-05-12 15:29:52 +03:00
Vyacheslav Egorov
3247aabf75 Sync to upstream/release/576 2023-05-12 15:15:01 +03:00
JohnnyMorganz
8d8c7974f5
Use correct globalScope in on demand type checker (#923)
Fixes #922 

Unsure if this needs to be put behind another FFlag, since the on demand
type checker fflag currently exists
2023-05-11 04:35:56 -07:00
Jan
45c6476d41
Add missing include for integer types (#925)
Closes #924

`uintptr_t` and `uint32` (and possibly more) were undefined because
`cstdint` was not included.
2023-05-09 11:42:13 -07:00
Andy Friesen
0014905b69
Luau.Analyze.CLI must link pthreads on Linux. (#920) 2023-05-05 15:41:46 -07:00
Andy Friesen
f7c780164d Add pthread as a link dependency to Luau.Analyze.CLI for Linux. 2023-05-05 15:33:05 -07:00
Andy Friesen
8453570658
Sync to upstream/release/575 (#919)
* `Luau.Analyze.CLI` now has experimental support for concurrent type
checking. Use the option `-jN` where `N` is the number of threads to
spawn.
* Improve typechecking performance by ~17% by making the function
`Luau::follow` much more efficient.
* Tighten up the type of `os.date`
* Removed `ParseOptions::allowTypeAnnotations` and
`ParseOptions::supportContinueStatement`

New solver

* Improve the reliability of function overload resolution
* More work toward supporting parallel type checking
* Fix a bug in inference of `==` and `~=` which would erroneously infer
that the operands were `boolean`
* Better error reporting when `for...in` loops are used incorrectly.

CodeGen

* Fix unwind registration when libunwind is used on Linux
* Fixed replaced IR instruction use count
* Convert X64 unwind info generation to standard prologue
* Implement A64 unwind info support for Dwarf2
* Live in/out data for linear blocks is now created
* Add side-exit VM register requirements to the IR dump
* Reuse ConstPropState between block chains 
* Remove redundant base update

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-05-05 14:52:49 -07:00
Andy Friesen
12c1edf6c6 This test fails on a64 so disable it for now. 2023-05-05 13:25:00 -07:00
Andy Friesen
95f0a04e06 Merge branch 'upstream' into merge 2023-05-05 12:57:26 -07:00
Andy Friesen
cc96f86c08 Merge branch 'master' into merge 2023-05-05 12:57:23 -07:00
Andy Friesen
716f63321a Sync to upstream/release/575 2023-05-05 12:57:12 -07:00
JohnnyMorganz
5891de6724
Fix grammar issues (#915)
- The type list for function declarations does not accept defaults, but
the grammar incorrectly allowed this.
- Allow the empty table type `{}`
- Allow composite types and types surrounded by parentheses (e.g.
`string??`, `(string)`, `(string | number) & boolean`)

---------

Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
2023-05-04 05:39:18 -07:00
JohnnyMorganz
e8c0550586
Improve the type for os.date using overloads (#874)
Improves the type of `os.date` based on its documentation
https://create.roblox.com/docs/reference/engine/libraries/os#date

We know that the DateTypeResult table is only returned when the format
string is either "*t" or "!*t", and otherwise its just a formatted
string. This changes the type to showcase this appropriately.

I put the more strict type first in the overload because the ordering
makes a difference

Note: the API docs used in studio
(https://github.com/MaximumADHD/Roblox-Client-Tracker/blob/roblox/api-docs/en-us.json)
may need to be updated to reflect the overloaded type instead.


![image](https://user-images.githubusercontent.com/19635171/226919224-e9e8703b-2729-44b0-93f6-434c01bda204.png)

It may be possible to add a lint of this too to provide analysis similar
to how `string.format` works with a FormatString error. Will leave that
for a different PR though
2023-05-01 08:09:05 -07:00
vegorov-rbx
4b267aa5c5
Sync to upstream/release/574 (#910)
* Added a limit on how many instructions the Compiler can safely produce
(reported by @TheGreatSageEqualToHeaven)

C++ API Changes:
* With work started on read-only and write-only properties,
`Property::type` member variable has been replaced with `TypeId type()`
and `setType(TypeId)` functions.
* New `LazyType` unwrap callback now has a `void` return type, all
that's required from the callback is to write into `unwrapped` field.

In our work on the new type solver, the following issues were fixed:

* Work has started to support https://github.com/Roblox/luau/pull/77 and
https://github.com/Roblox/luau/pull/79
* Refinements are no longer applied on l-values, removing some
false-positive errors
* Improved overload resolution against expected result type
* `Frontend::prepareModuleScope` now works in the new solver
* Cofinite strings are now comparable

And these are the changes in native code generation (JIT):

* Fixed MIN_NUM and MAX_NUM constant fold when one of the arguments is
NaN
* Added constant folding for number conversions and bit operations
* Value spilling and rematerialization is now supported on arm64
* Improved FASTCALL2K IR generation to support second argument constant
* Added value numbering and load/store propagation optimizations
* Added STORE_VECTOR on arm64, completing the IR lowering on this target
2023-04-28 12:55:13 -07:00
Vyacheslav Egorov
348f85cc1c Merge branch 'upstream' into merge 2023-04-28 14:57:08 +03:00
Vyacheslav Egorov
8bd3714566 Merge branch 'master' into merge 2023-04-28 14:56:34 +03:00
Vyacheslav Egorov
1c2ce0d731 Sync to upstream/release/574 2023-04-28 14:55:55 +03:00
JohnnyMorganz
8173491d86
Update grammar.md (#909)
Fix issues mentioned in https://github.com/Roblox/luau/issues/908
2023-04-24 14:21:26 -05:00
Andy Friesen
fe7621ee8c
Sync to upstream/release/573 (#903)
* Work toward affording parallel type checking
* The interface to `LazyType` has changed:
* `LazyType` now takes a second callback that is passed the `LazyType&`
itself. This new callback is responsible for populating the field
`TypeId LazyType::unwrapped`. Multithreaded implementations should
acquire a lock in this callback.
* Modules now retain their `humanReadableNames`. This reduces the number
of cases where type checking has to call back to a `ModuleResolver`.
* https://github.com/Roblox/luau/pull/902
* Add timing info to the Luau REPL compilation output

We've also fixed some bugs and crashes in the new solver as we march
toward readiness.
* Thread ICEs (Internal Compiler Errors) back to the Frontend properly
* Refinements are no longer applied to lvalues
* More miscellaneous stability improvements

Lots of activity in the new JIT engine:

* Implement register spilling/restore for A64
* Correct Luau IR value restore location tracking
* Fixed use-after-free in x86 register allocator spill restore
* Use btz for bit tests
* Finish branch assembly support for A64
* Codesize and performance improvements for A64
* The bit32 library has been implemented for arm and x64

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-04-21 15:14:26 -07:00
Andy Friesen
1126490e24 Merge branch 'upstream' into merge 2023-04-21 14:45:20 -07:00
Andy Friesen
a0f9842ba2 Merge branch 'master' into merge 2023-04-21 14:41:12 -07:00
Andy Friesen
d5cdb687e0 Sync to upstream/release/573 2023-04-21 14:41:03 -07:00
JohnnyMorganz
8ed808eb52
Add prefix and name location to AstTypeReference (#902)
Closes #901
2023-04-17 07:19:56 -07:00
vegorov-rbx
d141a5c48d
Sync to upstream/release/572 (#899)
* Fixed exported types not being suggested in autocomplete
* `T...` is now convertible to `...any` (Fixes
https://github.com/Roblox/luau/issues/767)
* Fixed issue with `T?` not being convertible to `T | T` or `T?`
(sometimes when internal pointer identity is different)
* Fixed potential crash in missing table key error suggestion to use a
similar existing key
* `lua_topointer` now returns a pointer for strings

C++ API Changes:
* `prepareModuleScope` callback has moved from TypeChecker to Frontend
* For LSPs, AstQuery functions (and `isWithinComment`) can be used
without full Frontend data

A lot of changes in our two experimental components as well.

In our work on the new type-solver, the following issues were fixed:
* Fixed table union and intersection indexing
* Correct custom type environments are now used
* Fixed issue with values of `free & number` type not accepted in
numeric operations

And these are the changes in native code generation (JIT):
* arm64 lowering is almost complete with support for 99% of IR commands
and all fastcalls
* Fixed x64 assembly encoding for extended byte registers
* More external x64 calls are aware of register allocator
* `math.min`/`math.max` with more than 2 arguments are now lowered to IR
as well
* Fixed correctness issues with `math` library calls with multiple
results in variadic context and with x64 register conflicts
* x64 register allocator learnt to restore values from VM memory instead
of always using stack spills
* x64 exception unwind information now supports multiple functions and
fixes function start offset in Dwarf2 info
2023-04-14 11:06:22 -07:00
Vyacheslav Egorov
33b95582ac Build fix 2023-04-14 16:46:13 +03:00
Vyacheslav Egorov
1ffbf6cd14 Merge branch 'upstream' into merge 2023-04-14 15:06:30 +03:00
Vyacheslav Egorov
70ef0fbc83 Merge branch 'master' into merge 2023-04-14 15:06:24 +03:00
Vyacheslav Egorov
5e771b87ae Sync to upstream/release/572 2023-04-14 15:05:27 +03:00
Petri Häkkinen
7345891f6b
Add lua_getuserdatadtor (#870)
Some userdata objects may need to support manual destruction in addition
to automatic GC. For example, files, threads, GPU resources and objects
with large external allocations.

With Lua, a finalizer can be _generically_ called by invoking the __gc
metamethod manually, but this is currently not possible with tagged
userdata in Luau because it's not possible to query the destructor
associated with an userdata. While it is possible to workaround this by
duplicating the destructor table locally on client side (*), it's more
convenient to deduplicate the data and get the destructor using the API
instead.

(*) Note: a separate destructor table for each VM may be required if the
VMs use different set of tags.

Implementation notes:

1. I first considered adding a typedef for lua_Destructor but
unfortunately there are two kinds of destructors, one with and one
without the lua_State* argument, so I decided against it at this point.
Maybe it should be added later if the destructor API is unified (by
dropping the Lua state pointer argument?).

2. For some reason the conformance test produced warning "qualifier
applied to function type has no meaning; ignored" on VS2017 (possibly
because the test framework does not like function pointers for some
reason?). I silenced this by pulling out the test expressions from those
CHECKs.
2023-04-11 12:46:55 -07:00