Commit graph

1075 commits

Author SHA1 Message Date
vegorov-rbx
443903aa00
Sync to upstream/release/615 (#1175)
# What's changed?

* Luau allocation scheme was changed to handle allocations in 513-1024
byte range internally without falling back to global allocator
* coroutine/thread creation no longer requires any global allocations,
making it up to 15% faster (vs libc malloc)
* table construction for 17-32 keys or 33-64 array elements is up to 30%
faster (vs libc malloc)

### New Type Solver

* Cyclic unary negation type families are reduced to `number` when
possible
* Class types are skipped when searching for free types in unifier to
improve performance
* Fixed issues with table type inference when metatables are present
* Improved inference of iteration loop types
* Fixed an issue with bidirectional inference of method calls
* Type simplification will now preserve error suppression markers

### Native Code Generation

* Fixed TAG_VECTOR skip optimization to not break instruction use counts
(broken optimization wasn't included in 614)
* Fixed missing side-effect when optimizing generic loop preparation
instruction

---

### Internal Contributors

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vighnesh <vvijay@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
2024-03-01 10:45:26 -08:00
Arseny Kapoulkine
cc51e616ce
CodeGen: Optimize vector ops for X64 when the source is computed (#1174)
With the TAG_VECTOR change, we can now confidently distinguish cases
when the .w component
contains TVECTOR tag from cases where it doesn't: loads and tag ops
produce the tag, whereas
other instructions don't.

We now take advantage of this fact and only apply vandps with a mask
when we need to.

It would be possible to use a positive filter (explicitly checking for
source coming from ADD_VEC
et al), but there are more instructions to check this way and this is
purely an optimization so
it is allowed to be conservative (as in, the cost of a mistake here is a
potential slowdown,
not a correctness issue).

Additionally, this change only performs vandps once when the arguments
are the same instead
of doing it twice.

On the function that computes a polynomial approximation this change
makes it ~20% faster on Zen4.
2024-03-01 03:32:43 -08:00
Arseny Kapoulkine
c9324853e5
luau-compile: Fix usage of vector-ctor without vector-lib (#1172)
When --vector-lib is not specified, CompileOptions::vectorLib was set to
an empty string. This resulted in the builtin matching not working,
since vectorLib must either be a null pointer or a pointer to a valid
global identifier.

---------

Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
2024-02-26 09:15:13 -08:00
Vighnesh-V
3b0e93bec9
Sync to upstream/release/614 (#1173)
# What's changed?
Add program argument passing to scripts run using the Luau REPL! You can
now pass `--program-args` (or shorthand `-a`) to the REPL which will
treat all remaining arguments as arguments to pass to executed scripts.
These values can be accessed through variadic argument expansion. You
can read these values like so:
```
local args = {...} -- gets you an array of all the arguments
```
For example if we run the following script like `luau test.lua -a test1
test2 test3`:
```
-- test.lua
print(...)
```
you should get the output:
```
test1 test2 test3
```

### Native Code Generation

* Improve A64 lowering for vector operations by using vector
instructions
* Fix lowering issue in IR value location tracking! 
- A developer reported a divergence between code run in the VM and
Native Code Generation which we have now fixed

### New Type Solver

* Apply substitution to type families, and emit new constraints to
reduce those further
* More progress on reducing comparison  (`lt/le`)type families
* Resolve two major sources of cyclic types in the new solver

### Miscellaneous
* Turned internal compiler errors (ICE's) into warnings and errors

-------
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-02-23 12:08:34 -08:00
Arseny Kapoulkine
80928acb92
CodeGen: Extract all vector tag patching into TAG_VECTOR (#1171)
Instead of patching the tag component with TVECTOR in every instruction
that produces a vector value, we now use a separate IR instruction to do
this. This reduces implementation redundancy, but more importantly
allows for a class of optimizations:

- NUM_TO_VECTOR previously patched the component unconditionally but the
result was used only in MUL/DIV_VEC instructions that ignore it anyway;
we can now remove this.

- ADD_VEC et al can now forward the source of TAG_VECTOR instruction of
either input; this shortens the latency chain and in the future could
allow us to generate optimal vector instruction sequence once the
temporary stores are marked as dead.

- In the future on X64, ADD_VEC et al will be able to analyze the input
instruction and remove tag masking conditionally. This is not part of
this PR as it requires a decision around expected FP environment and/or
the necessity of the existing masking to begin with.

I've also renamed NUM_TO_VECTOR to NUM_TO_VEC so that "VEC" always
refers to "3 float values" and for consistency with ADD/etc.

Note: ADD_VEC input forwarding is currently performed unconditionally;
it may or may not increase the spills that can't be reloaded from the
stack.

On A64 this makes the Taylor series computation a tiny bit faster
(11.3ns => 11.0ns) as it removes the redundant ins instructions along
the NUM_TO_VEC path. Curiously, the optimization of forwarding
TAG_VECTOR input to arithmetic instructions actually has a small penalty
as without it this PR runs at 10.9 ns. I don't know if this is a
property of the benchmark though, as I just noticed that in this
benchmark type inference actually fails to infer parts of the
computation as a vector op. If desired I will happily omit this part of
the change and we can explore that separately.
2024-02-21 07:06:11 -08:00
Arseny Kapoulkine
c5f4d973d7
Improve A64 lowering for vector operations by using vector instructions (#1164)
This change replaces scalar versions of vector opcodes for A64 with
actual vector instructions.

We take the approach similar to X64: patch last component with zero,
perform the math, patch last component with type tag. I'm hoping that in
the future the type tag will be placed separately (separate IR opcode?)
because right now chains of math operations result in excessive type tag
operations.

To patch the type tag without always keeping a mask in a register,
ins.4s instructions can be used; unfortunately it's only capable of
patching a register in-place, so we need an extra register copy in case
it's not last-use. Usually it's last-use so the patch is free; probably
with IR rework mentioned above all of this can be improved (e.g.
load-with-patch will never need to copy).

~It's not 100% clear if we *have* to patch type tag: Apple does preserve
denormals but we'd need to benchmark this to see if there's an actual
performance impact. But for now we're playing it safe.~

This was tested by running the conformance tests, and new opcode
implementations were checked by comparing the result with
https://armconverter.com/.

Performance testing is complicated by the fact that OSS Luau doesn't
support vector constructor out of the box, and other limitations of
codegen. I've hacked vector constructor/type into REPL and confirmed
that on a test that calls this function in a loop (not inlined):

```
function fma(a: vector, b: vector, c: vector)
        return a * b + c
end
```

... this PR improves performance by ~6% (note that probably most of the
overhead here is the call dispatch; I didn't want to brave testing a
more complex expression). The assembly for an individual operation
changes as follows:

Before:

```
#   %14 = MUL_VEC %12, %13                                    ; useCount: 2, lastUse: %22
 dup         s29,v31.s[0]
 dup         s28,v30.s[0]
 fmul        s29,s29,s28
 ins         v31.s[0],v29.s[0]
 dup         s29,v31.s[1]
 dup         s28,v30.s[1]
 fmul        s29,s29,s28
 ins         v31.s[1],v29.s[0]
 dup         s29,v31.s[2]
 dup         s28,v30.s[2]
 fmul        s29,s29,s28
 ins         v31.s[2],v29.s[0]
```

After:

```
#   %14 = MUL_VEC %12, %13                                    ; useCount: 2, lastUse: %22
 ins         v31.s[3],w31
 ins         v30.s[3],w31
 fmul        v31.4s,v31.4s,v30.4s
 movz        w17,#4
 ins         v31.s[3],w17
```

**edit** final form (see comments):

```
#   %14 = MUL_VEC %12, %13                                    ; useCount: 2, lastUse: %22
 fmul        v31.4s,v31.4s,v30.4s
 movz        w17,#4
 ins         v31.s[3],w17
```
2024-02-16 08:30:35 -08:00
vegorov-rbx
ea14e65ea0
Sync to upstream/release/613 (#1167)
# What's changed?

* Compiler now targets bytecode version 5 by default, this includes
support for vector type literals and sub/div opcodes with a constant on
lhs

### New Type Solver

* Normalizer type inhabitance check has been optimized
* Added ability to reduce cyclic `and`/`or` type families 

### Native Code Generation

* `CodeGen::compile` now returns more specific causes of a code
generation failure
* Fixed linking issues on platforms that don't support unwind frame data
registration

---

### Internal Contributors

Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vighnesh <vvijay@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
2024-02-15 18:04:39 -08:00
Andy Friesen
d6c2472f0c
Sync to upstream/release/612 (#1162)
New solver

* Fix bugs where bidirectional type inference would fail to take effect
at the proper stage.
* Improve inference of mutually recursive functions
* Fix crashes

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-02-09 09:51:12 -08:00
Alexander McCord
67ce75e870
Sync to upstream/release/611 (#1160)
# What's changed?

### Native Code Generation

* Fixed an UAF relating to reusing a hash key after a weak table has
undergone some GC.
* Fixed a bounds check on arm64 to allow access to the last byte of a
buffer.

### New Type Solver

* Type states now preserves error-suppression, i.e. `local x: any = 5`
and `x.foo` does not error.
* Made error-suppression logic in subtyping more accurate.
* Subtyping now knows how to reduce type families.
* Fixed function call overload resolution so that the return type
resolves to the correct overload.
* Fixed a case where we attempted to reduce irreducible type families a
few too many times, leading to duplicate errors.
* Type checker needs to type check annotations in function signatures to
be able to report errors relating to those annotations.
* Fixed an UAF from a pointer to stack-allocated data in Subtyping's
`explainReasonings`.

### Nonstrict Type Checker

* Fixed a crash when calling a checked function of the form `math.abs`
with an incorrect argument type.
* Fixed a crash when calling a checked function with a number of
arguments that did not exactly match the number of parameters required.

---

### Internal Contributors

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vighnesh <vvijay@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-02-02 13:32:42 -08:00
Arseny Kapoulkine
974963a870
Enable newly added M1 GHA CI (#1158)
GitHub just released support for M1 runners for open source projects via
GHA. It can be selected by using macos-14 OS to run on.

This is very valuable because until now, Luau wasn't tested in OSS CI on
Arm64, despite the presence of a fairly large volume of AArch64 specific
code courtesy of A64 codegen. This change fixes that.

In the future we could also expand codecov reporting as it seems to work
well but it needs a little bit of cleanup for the build in question so
let's start small.
2024-02-01 05:18:27 -08:00
Alex Orlenko
d409f7946d
Fix setting sandbox on Lua instance without strlib (#1156)
Currently calling `luaL_sandbox` on Lua instance without loaded strlib
causes crash (assertion).
It happens because inside `luaL_sandbox` there is no check that
metatable for strings is present.
2024-01-30 07:23:58 -08:00
JohnnyMorganz
c0b17daebd
Handle autocomplete in table when no initial character present (#1155)
Closes #685
2024-01-29 09:52:00 -08:00
aaron
9c588be16d
Sync to upstream/release/610 (#1154)
# What's changed?

* Check interrupt handler inside the pattern match engine to eliminate
potential for programs to hang during string library function execution.
* Allow iteration over table properties to pass the old type solver. 

### Native Code Generation

* Use in-place memory operands for math library operations on x64.
* Replace opaque bools with separate enum classes in IrDump to improve
code maintainability.
* Translate operations on inferred vectors to IR.
* Enable support for debugging native-compiled functions in Roblox
Studio.

### New Type Solver

* Rework type inference for boolean and string literals to introduce
bounded free types (bounded below by the singleton type, and above by
the primitive type) and reworked primitive type constraint to decide
which is the appropriate type for the literal.
* Introduce `FunctionCheckConstraint` to handle bidirectional
typechecking for function calls, pushing the expected parameter types
from the function onto the arguments.
* Introduce `union` and `intersect` type families to compute deferred
simplified unions and intersections to be employed by the constraint
generation logic in the new solver.
* Implement support for expanding the domain of local types in
`Unifier2`.
* Rework type inference for iteration variables bound by for in loops to
use local types.
* Change constraint blocking logic to use a set to prevent accidental
re-blocking.
* Add logic to detect missing return statements in functions.

### Internal Contributors

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vighnesh <vvijay@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-01-26 19:20:56 -08:00
vegorov-rbx
cdd1a380db
Sync to upstream/release/609 (#1150)
### What's changed?
* Syntax for [read-only and write-only
properties](https://github.com/luau-lang/rfcs/pull/15) is now parsed,
but is not yet supported in typechecking

### New Type Solver
* `keyof` and `rawkeyof` type operators have been updated to match final
text of the [RFC](https://github.com/luau-lang/rfcs/pull/16)
* Fixed issues with cyclic type families that were generated for mutable
loop variables

### Native Code Generation
* Fixed inference for number / vector operation that caused an
unnecessary VM assist

---
### Internal Contributors
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-01-19 10:04:46 -08:00
Aleksi
73360e5399
Fix missing include for std::exchange (#1147) 2024-01-16 04:41:40 -08:00
Vighnesh-V
f31232d301
Sync to upstream/release/608 (#1145)
# Old Solver:

- Fix a bug in the old solver where a user could use the keyword
`typeof` as the name of a type alias.
- Fix stringification of scientific notation to omit a trailing decimal
place when not followed by a digit e.g. `1.e+20` -> `1e+20`
# New Solver
- Continuing work on the New non-strict mode
- Introduce `keyof` and `rawkeyof` type function for acquiring the type
of all keys in a table or class
(https://github.com/luau-lang/rfcs/pull/16)

---
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-01-12 14:25:27 -08:00
JohnnyMorganz
c0f5538947
Clone TableType typeLocation (#1136)
This was missed in #1046
2024-01-08 06:25:27 -08:00
vegorov-rbx
e76802f2ce
Add a prefix to TaggedLuData flag to have it auto-enabled in flags ON mode (#1130) 2023-12-15 13:51:14 -08:00
aaron
ff502f0943
Sync to upstream/release/607 (#1131)
# What's changed?

* Fix up the `std::iterator_traits` definitions for some Luau data
structures.
* Replace some of the usages of `std::unordered_set` and
`std::unordered_map` with Luau-provided data structures to increase
performance and reduce overall number of heap allocations.
* Update some of the documentation links in comments throughout the
codebase to correctly point to the moved repository.
* Expanded JSON encoder for AST to support singleton types.
* Fixed a bug in `luau-analyze` where exceptions in the last module
being checked during multithreaded analysis would not be rethrown.

### New type solver

* Introduce a `refine` type family to handle deferred refinements during
type inference, replacing the old `RefineConstraint`.
* Continued work on the implementation of type states, fixing some known
bugs/blockers.
* Added support for variadic functions in new non-strict mode, enabling
broader support for builtins and the Roblox API.

### Internal Contributors

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Vighnesh <vvijay@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-12-15 13:29:06 -08:00
Petri Häkkinen
2173938eb0
Add tagged lightuserdata (#1087)
This change adds support for tagged lightuserdata and optional custom
typenames for lightuserdata.

Background: Lightuserdata is an efficient representation for many kinds
of unmanaged handles and resources in a game engine. However, currently
the VM only supports one kind of lightuserdata, which makes it
problematic in practice. For example, it's not possible to distinguish
between different kinds of lightuserdata in Lua bindings, which can lead
to unsafe practices and even crashes when a wrong kind of lightuserdata
is passed to a binding function. Tagged lightuserdata work similarly to
tagged userdata, i.e. they allow checking the tag quickly using
lua_tolightuserdatatagged (or lua_lightuserdatatag).

The tag is stored in the 'extra' field of TValue so it will add no cost
to the (untagged) lightuserdata type.

Alternatives would be to use full userdata values or use bitpacking to
embed type information into lightuserdata on application level.
Unfortunately these options are not that great in practice: full
userdata have major performance implications and bitpacking fails in
cases where full 64 bits are already used (e.g. pointers or 64-bit
hashes).

Lightuserdata names are not strictly necessary but they are rather
convenient when debugging Lua code. More precise error messages and
tostring returning more specific typename are useful to have in practice
(e.g. "resource" or "entity" instead of the more generic "userdata").

Impl note: I did not add support for renaming tags in
lua_setlightuserdataname as I'm not sure if it's possible to free fixed
strings. If it's simple enough, maybe we should allow renaming (although
I can't think of a specific need for it)?

---------

Co-authored-by: Petri Häkkinen <petrih@rmd.remedy.fi>
2023-12-14 15:05:51 -08:00
vegorov-rbx
c26d820902
Sync to upstream/release/606 (#1127)
New Solver
* Improvements to data flow analysis

Native Code Generation
* Block limit is now per-function instead of per-module

Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-12-08 13:50:16 -08:00
Vighnesh-V
2ea7193777
Resolve RequireByString test suite failures on Ubuntu CI (#1124)
This PR resolves the RequireByString test suite failures in CI on
Ubuntu. The issue was that paths on linux are case sensitive and thus
MacOS/Windows machines simply behaved as if the paths to the test files
were case-insensitive, but the linux machine was unable to find the test
files. Those tests were `#ifdef'd` out, and have been un `#ifdef'd` to
demonstrate these changes work.
2023-12-05 14:31:02 -08:00
Vyacheslav Egorov
aafea36235
Fixed the backwards compatible benchmark support library require (#1125)
Previous benchmark require fix wasn't actually working correctly for the
old style require (or running in Lua).
2023-12-04 12:48:31 -08:00
menarulalam
765ac75fe9
Update ExperimentalFlags.h (#1123)
Add LuauUpdatedRequireByStringSemantics to experimental flags.
2023-12-04 14:26:39 -05:00
Vyacheslav Egorov
ce1800746b
Fix 'require' in benchmarks to work with new relative system (#1120)
Old style require is now called with `pcall` to support comparing
against Lua.
New style require is now a third option.

Edit: this will be a temporary solution until the 'paths' support in
.luaurc is fixed.
2023-12-02 14:20:54 -08:00
Vyacheslav Egorov
6413ea8574
Only build fuzzer with CMake if 3.26 is available (#1119)
`cmake_minimum_required` is a blocking check and raising required
minimal CMake version only for the fuzzer is not good for the users
(<3.19 was supported before, but not sure which exact one)
2023-12-02 13:04:25 -08:00
Vighnesh-V
c755875479
Sync to upstream/release/605 (#1118)
- Implemented [Require by String with Relative
Paths](https://github.com/luau-lang/rfcs/blob/master/docs/new-require-by-string-semantics.md)
RFC
- Implemented [Require by String with
Aliases](https://github.com/luau-lang/rfcs/blob/master/docs/require-by-string-aliases.md)
RFC with support for `paths` and `alias` arrays in .luarc
- Added SUBRK and DIVRK bytecode instructions to speed up
constant-number and constant/number operations
- Added `--vector-lib`, `--vector-ctor` and `--vector-type` options to
luau-compile to support code with vectors
 
New Solver
- Correctness fixes to subtyping
- Improvements to dataflow analysis

Native Code Generation
- Added bytecode analysis pass to predict type tags used in operations
- Fixed rare cases of numerical loops being generated without an
interrupt instruction
- Restored optimization data propagation into the linear block
- Duplicate buffer length checks are optimized away

Miscellaneous
- Small performance improvements to new non-strict mode
- Introduced more scripts for fuzzing Luau and processing the results,
including fuzzer build support for CMake

Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-12-01 23:46:57 -08:00
Arseny Kapoulkine
89b437bb4e
Add SUBRK and DIVRK bytecode instructions to bytecode v5 (#1115)
Right now, we can compile R\*K for all arithmetic instructions, but K\*R
gets compiled into two instructions (LOADN/LOADK + arithmetic opcode).

This is problematic since it leads to reduced performance for some code.
However, we'd like to avoid adding reverse variants of ADDK et al for
all opcodes to avoid the increase in I$ footprint for interpreter.

Looking at the arithmetic instructions, % and // don't have interesting
use cases for K\*V; ^ is sometimes used with constant on the left hand
side but this would need to call pow() by necessity in all cases so it
would be slow regardless of the dispatch overhead. This leaves the four
basic arithmetic operations.

For + and \*, we can implement a compiler-side optimization in the
future that transforms K\*R to R\*K automatically. This could either be
done unconditionally at -O2, or conditionally based on the type of the
value (driven by type annotations / inference) -- this technically
changes behavior in presence of metamethods, although it might be
sensible to just always do this because non-commutative +/* are evil.

However, for - and / it is impossible for the compiler to optimize this
in the future, so we need dedicated opcodes. This only increases the
interpreter size by ~300 bytes (~1.5%) on X64.

This makes spectral-norm and math-partial-sums 6% faster; maybe more
importantly, voxelgen gets 1.5% faster (so this change does have
real-world impact).

To avoid the proliferation of bytecode versions this change piggybacks
on the bytecode version bump that was just made in 604 for vector
constants; we would still be able to enable these independently but
we'll consider v5 complete when both are enabled.

Related: #626

---------

Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
2023-11-28 07:35:01 -08:00
Arseny Kapoulkine
7fb7f4382d
Use ubuntu-latest on GHA when possible (#1112)
We need ubuntu-20.04 for coverage analysis (clang after 10 doesn't seem
to properly interact with gcov version used for codecov) and for
releases (to produce binaries targeting earlier glibc).

However, we still should be verifying that Luau builds on latest,
because newer toolchains have stricter standard library headers and/or
warnings; without this we're at risk of constantly regressing the build
for packaging or external applications.

Note that release.yml can probably just be deleted but for now we simply
adjust it.
2023-11-27 03:24:57 -08:00
vegorov-rbx
1cda8304bf
Add missing include file to BytecodeSummary.h (#1110) 2023-11-21 08:28:54 -08:00
JohnnyMorganz
ae53051814
Include module name for definitions files (#861)
Closes #818 

We set the `moduleName` of the source module to the provided
`packageName`.
This then populates the relevant `definitionModuleName`'s on
CTV/FTV/TTVs, so it allows us to lookup where they originated from.

The one place I can see this having an impact inside of Luau code is the
following:

1fa8311a18/Analysis/src/Error.cpp (L70-L90)
Which will potentially change they way error messages are formatted.
Should there be a way to exclude definition files when calling
`getDefinitionModuleName`, so we can preserve this behaviour?
2023-11-21 08:12:18 -08:00
Andy Friesen
74c532053f
Sync to upstream/release/604 (#1106)
New Solver

* New algorithm for inferring the types of locals that have no
annotations. This
algorithm is very conservative by default, but is augmented with some
control
  flow awareness to handle most common scenarios.
* Fix bugs in type inference of tables
* Improve performance of by switching out standard C++ containers for
`DenseHashMap`
* Infrastructure to support clearer error messages in strict mode

Native Code Generation

* Fix a lowering issue with buffer.writeu8 and 0x80-0xff values: A
constant
  argument wasn't truncated to the target type range and that causes an
  assertion failure in `build.mov`.
* Store full lightuserdata value in loop iteration protocol lowering
* Add analysis to compute function bytecode distribution
* This includes a class to analyze the bytecode operator distribution
per
function and a CLI tool that produces a JSON report. See the new cmake
      target `Luau.Bytecode.CLI`

---------

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-17 10:46:18 -08:00
Petri Häkkinen
298cd70154
Optimize vector literals by storing them in the constant table (#1096)
With this optimization, built-in vector constructor calls with 3/4 arguments are detected by the compiler and turned into vector constants when the arguments are constant numbers.
Requires optimization level 2 because built-ins are not folded otherwise by the compiler.
Bytecode version is bumped because of the new constant type, but old bytecode versions can still be loaded.

The following synthetic benchmark shows ~6.6x improvement.
```
local v
for i = 1, 10000000 do
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
	v = vector(1, 2, 3)
end
```

Also tried a more real world scenario and could see a few percent improvement.

Added a new fast flag LuauVectorLiterals for enabling the feature.

---------

Co-authored-by: Petri Häkkinen <petrih@rmd.remedy.fi>
Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
2023-11-17 04:54:32 -08:00
Kostadin
0492ecffdf
Add #include <algorithm> to fix building with gcc 14 (#1104)
With gcc 14 some C++ Standard Library headers have been changed to no
longer include other headers that were used internally by the library.
In luau's case it is the `<algorithm>` header.
2023-11-16 11:51:16 -08:00
Arseny Kapoulkine
d2059dd50d
Update codecov.yml
Make status checks informational
2023-11-15 06:30:40 -08:00
Arseny Kapoulkine
64d2d0fb76
Update README.md
Use rfcs.luau-lang.org for the .luaurc link
2023-11-13 07:41:15 -08:00
JohnnyMorganz
4b2af900c2
Fix link to .luaurc info in README (#1101) 2023-11-12 13:48:41 -08:00
Alexander McCord
c2ba1058c3
Sync to upstream/release/603 (#1097)
# What's changed?

- Record the location of properties for table types (closes #802)
- Implement stricter UTF-8 validations as per the RFC
(https://github.com/luau-lang/rfcs/pull/1)
- Implement `buffer` as a new type in both the old and new solvers.
- Changed errors produced by some `buffer` builtins to be a bit more
generic to avoid platform-dependent error messages.
- Fixed a bug where `Unifier` would copy some persistent types, tripping
some internal assertions.
- Type checking rules on relational operators is now a little bit more
lax.
- Improve dead code elimination for some `if` statements with complex
always-false conditions

## New type solver

- Dataflow analysis now generates phi nodes on exit of branches.
- Dataflow analysis avoids producing a new definition for locals or
properties that are not owned by that loop.
- If a function parameter has been constrained to `never`, report errors
at all uses of that parameter within that function.
- Switch to using the new `Luau::Set` to replace `std::unordered_set` to
alleviate some poor allocation characteristics which was negatively
affecting overall performance.
- Subtyping can now report many failing reasons instead of just the
first one that we happened to find during the test.
- Subtyping now also report reasons for type pack mismatches.
- When visiting `if` statements or expressions, the resulting context
are the common terms in both branches.

## Native codegen

- Implement support for `buffer` builtins to its IR for x64 and A64.
- Optimized `table.insert` by not inserting a table barrier if it is
fastcalled with a constant.

## Internal Contributors

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Arseny Kapoulkine <arseny@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-11-10 13:10:07 -08:00
aaron
7105c81579
Sync to upstream/release/602 (#1089)
# What's changed?

* Fixed a bug in type cloning by maintaining persistent types.
* We now parse imprecise integer literals to report the imprecision as a
warning to developers.
* Add a compiler flag to specify the name of the statistics output file.

### New type solver

* Renamed `ConstraintGraphBuilder` to `ConstraintGenerator`
* LValues now take into account the type being assigned during
constraint generation.
* Normalization performance has been improved by 33% by replacing the an
internal usage of `std::unordered_set` with `DenseHashMap`.
* Normalization now has a helper to identify types that are equivalent
to `unknown`, which is being used to fix some bugs in subtyping.
* Uses of the old unifier in the new type solver have been eliminated.
* Improved error explanations for subtyping errors in `TypeChecker2`.

### Native code generation

* Expanded some of the statistics recorded during compilation to include
the number of instructions and blocks.
* Introduce instruction and block count limiters for controlling what
bytecode is translated into native code.
* Implement code generation for byteswap instruction.

### Internal Contributors

Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Alexander McCord <amccord@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Lily Brown <lbrown@roblox.com>
2023-11-03 16:45:04 -07:00
JohnnyMorganz
1a9159daff
Record table type alias property locations (#1046)
For table type aliases, records the location of the property in the
alias declaration.

This is an alternate solution to the particular case noted in #802.
Instead of tracking the type alias definition for FTVs, it tracks it for
the encompassing property instead.

Note that we still don't have positions in the following case:

```
type Func = () -> ()
```

Closes #802 (Although not completely...)
2023-11-02 09:14:51 -07:00
Arseny Kapoulkine
5622474b6a Merge remote-tracking branch 'origin/merge' 2023-11-01 18:11:17 -07:00
Arseny Kapoulkine
b647288375
Delete rfcs directory
This folder is now hosted in a separate repository https://github.com/luau-lang/rfcs (see https://github.com/luau-lang/luau/issues/1074 for details)
2023-10-30 10:49:36 -07:00
Arseny Kapoulkine
88eec1da37
Update CONTRIBUTING.md
Fix the guide to point to docs and rfcs repositories for associated changes
2023-10-30 09:16:59 -07:00
Arseny Kapoulkine
6d564abf81
Update README.md
Correct links and language to docs and RFCs repository and mention usage in non-Roblox games.
2023-10-30 09:02:22 -07:00
Arseny Kapoulkine
45f4b87991 Attempt to disable codecov patch status check 2023-10-27 17:11:17 -07:00
Lily Brown
e5ec0cdff3
Sync to upstream/release/601 (#1084)
## What's changed

- `bit32.byteswap` added
([RFC](4f543ec23b/docs/function-bit32-byteswap.md))
- Buffer library implementation
([RFC](4f543ec23b/docs/type-byte-buffer.md))
- Fixed a missing `stdint.h` include
- Fixed parser limiter for recursive type annotations being kind of
weird (fixes #645)

### Native Codegen
- Fixed a pair of issues when lowering `bit32.extract`
- Fixed a narrow edge case that could result in an infinite loop without
an interruption
- Fixed a negative array out-of-bounds access issue
- Temporarily reverted linear block predecessor value propagation

### New type solver
- We now type check assignments to annotated variables
- Fixed some test cases under local type inference
- Moved `isPending` checks for type families to improve performance
- Optimized our process for testing if a free type is sufficiently
solved
- Removed "none ptr" from lea instruction disassembly logging

### Build system & tooling
- CMake configuration now validates dependencies to maintain separation
between components
- Improvements to the fuzzer coverage
- Deduplicator for fuzzed callstacks

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
2023-10-27 14:18:41 -07:00
Lily Brown
98217437ec perhaps this works better 2023-10-27 13:42:42 -07:00
Lily Brown
9216e4cb27 attempt allowing a more complex expression here 2023-10-27 13:22:39 -07:00
Lily Brown
592469373b Merge branch 'upstream' into merge 2023-10-27 13:04:04 -07:00
Lily Brown
a4ceaa10ec Merge branch 'master' into merge 2023-10-27 12:34:25 -07:00