Commit graph

684 commits

Author SHA1 Message Date
vegorov-rbx
140e5a1495
Sync to upstream/release/566 (#853)
* Fixed incorrect lexeme generated for string parts in the middle of an
interpolated string (Fixes https://github.com/Roblox/luau/issues/744)
* DeprecatedApi lint can report some issues without type inference
information
* Fixed performance of autocomplete requests when suggestions have large
intersection types (Solves
https://github.com/Roblox/luau/discussions/847)
* Marked `table.getn`/`foreach`/`foreachi` as deprecated ([RFC:
Deprecate
table.getn/foreach/foreachi](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md))
* With -O2 optimization level, we now optimize builtin calls based on
known argument/return count.
Note that this change can be observable if `getfenv/setfenv` is used to
substitute a builtin, especially if arity is different.
Fastcall heavy tests show a 1-2% improvement.
* Luau can now be built with clang-cl (Fixes
https://github.com/Roblox/luau/issues/736)

We also made many improvements to our experimental components.

For our new type solver:
* Overhauled data flow analysis system, fixed issues with 'repeat'
loops, global variables and type annotations
* Type refinements now work on generic table indexing with a string
literal
* Type refinements will properly track potentially 'nil' values (like
t[x] for a missing key) and their further refinements
* Internal top table type is now isomorphic to `{}` which fixes issues
when `typeof(v) == 'table'` type refinement is handled
* References to non-existent types in type annotations no longer resolve
to 'error' type like in old solver
* Improved handling of class unions in property access expressions
* Fixed default type packs
* Unsealed tables can now have metatables
* Restored expected types for function arguments

And for native code generation:
* Added min and max IR instructions mapping to vminsd/vmaxsd on x64
* We now speculatively extract direct execution fast-paths based on
expected types of expressions which provides better optimization
opportunities inside a single basic block
* Translated existing math fastcalls to IR form to improve tag guard
removal and constant propagation
2023-03-03 12:21:14 -08:00
Arseny Kapoulkine
48172dd5b1
Update SECURITY.md
Note that native code gen is currently exempt from any security guarantees as it's a pre-production R&D component right now. This will change in the future as we deploy it to production.
2023-03-01 14:40:40 -08:00
Arseny Kapoulkine
6601c41bff
Correct table.getn/foreach deprecation RFC (#848)
It looks like all three functions actually have been deprecated in Lua
5.1, and removed in Lua 5.2.

We do not plan to remove them to retain backwards compatibility, but the
RFC should be more precise.
2023-02-27 12:30:33 -08:00
Arseny Kapoulkine
aef99ae0d4
docs: Add deprecation notice to getn/foreach/foreachi (#849)
RFC:
https://github.com/Roblox/luau/blob/master/rfcs/deprecate-table-getn-foreach.md
2023-02-27 12:30:23 -08:00
Andy Friesen
d2ab5df62b
Sync to upstream/release/565 (#845)
We've made a few small changes to reduce the amount of stack we use when
typechecking nested method calls (eg `foo:bar():baz():quux()`).

We've also fixed a small bytecode compiler issue that caused us to emit
redundant jump instructions in code that conditionally uses `break` or
`continue`.

On the new solver, we've switched to a new, better way to handle
augmentations to unsealed tables. We've also made some substantial
improvements to type inference and error reporting on function calls.
These things should both be on par with the old solver now.

The main improvements to the native code generator have been elimination
of some redundant type tag checks. Also, we are starting to inline
particular fastcalls directly to IR.

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-02-24 13:49:38 -08:00
B. Gibbons
fbd93cbc03
docs: update readme installation guide (#792)
1. Added information about Homebrew installation (based off this
information [here](https://formulae.brew.sh/formula/luau#default))
2. Added details related to installing binary files that are downloaded
3. Rearranged and added titles for easier reading

Fixes #791.

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
2023-02-23 13:08:45 -08:00
vegorov-rbx
b570ff0a37
Sync to upstream/release/564 (#841)
This week we only have updates to new type solver and JIT. Both projects
are still in the process of being built out. Neither are ready for
general use yet.

In the new solver, we fixed issues with recursive type aliases.
Duplicated type parameters are once again reported, exported types are
being recorder and function argument names are placed inside function
types.
We also made improvements to restore parts of bidirectional type
tracking.

On native code generation side, namecall instruction lowering was fixed,
we fixed inconsistencies in IR command definitions and added utility
function to help with constant folding.
2023-02-17 15:41:51 -08:00
Arseny Kapoulkine
2e5f95ca58
Update STATUS.md 2023-02-14 12:25:58 -08:00
Arseny Kapoulkine
8f37fc6dfc
RFC: Deprecate table.getn/foreach/foreachi (#715)
[Rendered](https://github.com/Roblox/luau/blob/zeux-rfc-deptable/rfcs/deprecate-table-getn-foreach.md)
2023-02-14 12:25:12 -08:00
Arseny Kapoulkine
742702a17d
Update lower-bounds-calculation.md
This RFC is abandoned; for posterity we will keep this in the tree for now, as it's likely that it represents a subset of the future set of LTI rules
2023-02-14 12:23:54 -08:00
Andy Friesen
c5089def6e
Sync to upstream/release/563 (#833)
* Fix a bug where reading a property from an unsealed table caused
inference to improperly infer the existence of that property.
* Fix #827

We have also made a lot of progress on the new solver and the JIT. Both
projects are still in the process of being built out. Neither are ready
for general use yet.

We are mostly working to tighten up how the new solver handles
refinements and updates to unsealed tables to bring it up to the same
level as the old solver.

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-02-10 11:40:38 -08:00
Epix
18a1dc3440
Fix comments on jump instructions with AUX (#808)
It seems like for jump instructions that include an AUX 32-bit word,
that index is included as part of the jump offset.
2023-02-09 06:12:58 -08:00
vegorov-rbx
c5555c6573
News about string interpolation feature and a syntax page update (#829)
News can be viewed
[here](https://vegorov-rbx.github.io/luau/2023/02/02/luau-string-interpolation.html)
And syntax can be viewed
[here](https://vegorov-rbx.github.io/luau/syntax#string-interpolation)

Note that link appears to be broken in the news section. It goes to
`https://vegorov-rbx.github.io/syntax#string-interpolation` instead of
`https://vegorov-rbx.github.io/luau/syntax#string-interpolation`
The link I use in the source is `/syntax#string-interpolation` but the
root 'offset' is different on Luau website (doesn't have extra 'luau'
folder) and on my GitHub pages.
Hope this is ok. But if GitHub page masters know how to avoid this,
please let me know.

---------

Co-authored-by: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com>
Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com>
2023-02-03 11:41:13 -08:00
vegorov-rbx
62483d40f0
Sync to upstream/release/562 (#828)
* Fixed rare use-after-free in analysis during table unification

A lot of work these past months went into two new Luau components:
* A near full rewrite of the typechecker using a new deferred constraint
resolution system
* Native code generation for AoT/JiT compilation of VM bytecode into x64
(avx)/arm64 instructions

Both of these components are far from finished and we don't provide
documentation on building and using them at this point.
However, curious community members expressed interest in learning about
changes that go into these components each week, so we are now listing
them here in the 'sync' pull request descriptions.

---
New typechecker can be enabled by setting
DebugLuauDeferredConstraintResolution flag to 'true'.
It is considered unstable right now, so try it at your own risk.
Even though it already provides better type inference than the current
one in some cases, our main goal right now is to reach feature parity
with current typechecker.
Features which improve over the capabilities of the current typechecker
are marked as '(NEW)'.

Changes to new typechecker:
* Regular for loop index and parameters are now typechecked
* Invalid type annotations on local variables are ignored to improve
autocomplete
* Fixed missing autocomplete type suggestions for function arguments
* Type reduction is now performed to produce simpler types to be
presented to the user (error messages, custom LSPs)
* Internally, complex types like '((number | string) & ~(false?)) |
string' can be produced, which is just 'string | number' when simplified
* Fixed spots where support for unknown and never types was missing
* (NEW) Length operator '#' is now valid to use on top table type, this
type comes up when doing typeof(x) == "table" guards and isn't available
in current typechecker

---
Changes to native code generation:
* Additional math library fast calls are now lowered to x64: math.ldexp,
math.round, math.frexp, math.modf, math.sign and math.clamp
2023-02-03 11:26:13 -08:00
Andy Friesen
f763f4c948
Sync to upstream/release/561 (#820)
* Fix a potential debugger crash by adding checks for invalid stack
index values

---------

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-01-27 14:28:31 -08:00
vegorov-rbx
4a2e8013c7
Sync to upstream/release/560 (#810)
* For autocomplete, additional information is included in Scope for type
alias name locations and names of imported modules
* Improved autocomplete suggestions in 'for' and 'while' loop headers
* String match functions return types are now optional strings and
numbers because match is not guaranteed at runtime
* Fixed build issue on gcc 11 and up (Fixes
https://github.com/Roblox/luau/issues/806)
2023-01-20 12:27:03 -08:00
Harold Cindy
729bc44729
Fix lua_*upvalue() when upvalue names aren't in debug info (#787)
`lua_getupvalue()` and `lua_setupvalue()` don't behave as expected when
working with Lua closure whose `Proto` has no debug info. The code
currently uses `sizeupvalues` to do bounds checking of upvalue indices,
but that's the size of the upvalue _names_ array. It will always be `0`
if the `Proto` doesn't have debug info.

This uses `nups` instead, and just returns `""` as the upvalue name if
we don't have one, same as for C closures.

Co-authored-by: Harold Cindy <HaroldCindy@users.noreply.github.com>
2023-01-18 06:00:13 -08:00
Andy Friesen
a5c6a38b10
Sync to upstream/release/559 (#804)
* Fix autocompletion of if-then-else expressions
* Fix a potential crash surrounding improper use of `%*` in a string
format specifier
* All Python scripts now invoke Python via `python3` rather than
`python`.
* Improved error handling for string interpolation with too many
arguments.

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-01-13 14:10:01 -08:00
JohnnyMorganz
0af10417cd
Attach definition location to TableType (#801)
Having the location where the table type is defined is useful for
tracking other info such as documentation comments, or pointing back
through "Go To Definition" functionality etc.

This adds in the required info
2023-01-12 06:21:25 -08:00
JohnnyMorganz
86494918f5
Pass string content to autocomplete callback (#800)
Closes #718 

We pass an extra `contents` parameter to the string callback function to
provide contextual information for autocomplete.

Because we support both string literals and simple interpolated strings,
we pass it through as a `std::string` value.

This is a breaking change
2023-01-11 08:28:11 -08:00
vegorov-rbx
be52bd91e4
Sync to upstream/release/558 (#796)
* Fixed garbage data in module scopes when type graph is not retained
* LOP_MOVE with the same source and target registers is no longer
generated (Fixes https://github.com/Roblox/luau/issues/793)
2023-01-06 13:14:35 -08:00
vegorov-rbx
685ca02a30
Keep using ubuntu-20.04 in GitHub Actions (and clang++-10 in coverage) (#795)
### Problem
ubuntu-latest was updated to 22.04 which removes clang++-10 we used for
coverage stats and creates a pre-compiled binary that requires a glibc
upgrade https://github.com/Roblox/luau/issues/773

### Solution
Pin to ubuntu-20.04 using multi-value matrix configurations.
In coverage configuration, we use clang++-10 once again.
2023-01-06 12:17:25 -08:00
vegorov-rbx
75a2e95714
Sync to upstream/release/557 (#794)
* Fixed unions of `nil` types displaying as `?`
* Internal normalization now handles class types which can make
previously failing (incorrectly) sub-typing checks to succeed
2023-01-04 12:53:17 -08:00
Andy Friesen
fb2f146123
Sync to upstream/release/556 (#782)
* The AST JSON encoder will now stringify infinity and NaN according to the JSON5 spec using the tokens `Infinity`, `-Infinity`, and `NaN`.
* Improve autocompletion of table keys if the type of that key is a union of string singletons.
2022-12-09 11:57:01 -08:00
Alex Orlenko
e9d4ee7a33
Make pseudo-indices relative to LUAI_MAXCSTACK (#766)
This is useful in particular if redefine `LUAI_MAXCSTACK` to a higher
value than the current one (8000).
Ie. passing `-D LUAI_MAXCSTACK=1000000` would not work as overlaps with
`LUA_REGISTRYINDEX` and below.
2022-12-06 11:20:24 -08:00
vegorov-rbx
4e6ae32625
Luau Recap: November 2022 (#764)
Co-authored-by: Andy Friesen <andy.friesen@gmail.com>
2022-12-02 10:23:20 -08:00
vegorov-rbx
59ae47db43
Sync to upstream/release/555 (#768)
* Type mismatch errors now mention if unification failed in covariant or
invariant context, to explain why sometimes derived class can't be
converted to base class or why `T` can't be converted into `T?` and so
on
* Class type indexing is no longer an error in non-strict mode (still an
error in strict mode)
* Fixed cyclic type packs not being displayed in the type
* Added an error when unrelated types are compared with `==`/`~=`
* Fixed false positive errors involving sub-type tests an `never` type
* Fixed miscompilation of multiple assignment statements (Fixes
https://github.com/Roblox/luau/issues/754)
* Type inference stability improvements
2022-12-02 10:09:59 -08:00
vegorov-rbx
91302d1d4c
Fix coverage github action by using clang++ without a specific version (#769)
Github fails to find clang++-10.
Since we install llvm without a specific version number, we shouldn't
use clang++-10 with a version number.

I tried installing llvm-10, but that package is not available on github
(any more?).
2022-12-02 10:08:55 -08:00
JohnnyMorganz
9095fc4b83
Support __call on class type vars (#762)
Currently, the metatable of a class type var is not correctly checked to
see if it is callable (i.e. has a `__call` metatable).
We resolve this issue by checking the metatable in `checkCallOverload`

Fixes #756

Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
2022-11-28 10:02:41 -08:00
JohnnyMorganz
7dbe47f4dd
Handle cyclically referenced declared classes (#729)
Closes #631 

Performs a "two pass" approach, where we first call `prototype` on the
class definition.
The prototype phase checks for errors and creates a base CTV for the
class, which other types can then reference.

The second `check` phase fills it out with all the defined
properties/methods

Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
2022-11-28 05:25:44 -08:00
Andy Friesen
95d9c6d194
Sync to upstream/release/553 (#751)
* Autocomplete support for interpolated strings.
* Improved parse errors in various situations involving interpolated
strings.
2022-11-18 11:47:21 -08:00
boyned//Kampfkarren
aa7c64517c
Fix string interpolation autocomplete and location (#748)
String interpolation autocomplete was not working and was just using
default autocomplete. This fixes it so that inside a string it is
treated as normal, and inside an expression it suggests expression level
autocomplete.

Also fixes the range, which was previously just the first lexeme.
2022-11-16 10:15:01 -08:00
vegorov-rbx
816e41a8f2
Sync to upstream/release/553 (#742)
* Type inference of `a and b` and `a or b` has been improved (Fixes
https://github.com/Roblox/luau/issues/730)
* Improved error message when `for ... in x` loop iterates over a value
that could be 'nil'
* Return type of `next` not includes 'nil' (Fixes
https://github.com/Roblox/luau/issues/706)
* Improved type inference of `string` type
* Luau library table type names are now reported as `typeof(string)`/etc
instead of just `string` which was misleading
* Added parsing error when optional type alias type parameter wasn't
provided after `=` token
* Improved tagged union type refinement in conditional expressions, type
in `else` branch should no longer include previously handled union
options
2022-11-10 14:53:13 -08:00
Arseny Kapoulkine
0f04d521e6
Add "Luau origins and evolution" post (#737) 2022-11-04 15:26:37 -07:00
Andy Friesen
c33700e473
Sync to upstream/release/552 (#735)
* Reduce the stack utilization of type checking.
* Improve the error message that's reported when a delimiting comma is
missing from a table literal. eg
```lua
local t = {
    first = 1
    second = 2
}```
2022-11-04 10:33:22 -07:00
Alan Jeffrey
e43a9e927c
Added Sep-Oct 2022 Luau Recap (#726) 2022-11-01 12:02:03 -05:00
boyned//Kampfkarren
e37eb3c778
Fix { range to be within the interpolated string (#728)
Corrects `{` range to be inside the interpolated string, needed for syntax highlighting.
2022-10-28 12:22:26 -07:00
vegorov-rbx
a6b5051edc
Sync to upstream/release/551 (#727)
* https://github.com/Roblox/luau/pull/719
* Improved `Failed to unify type packs` error message to be reported as
`Type pack 'X' could not be converted into 'Y'`
* https://github.com/Roblox/luau/pull/722
* 1% reduction in executed instruction count by removing a check in fast
call dispatch
* Additional fixes to reported error location of OOM errors in VM
* Improve `math.sqrt`, `math.floor` and `math.ceil` performance on
additional compilers and platforms (1-2% geomean improvement including
8-9% on math-cordic)
* All thrown exceptions by Luau analysis are derived from
`Luau::InternalCompilerError`
* When a call site has fewer arguments than required, error now reports
the location of the function name instead of the argument to the
function
* https://github.com/Roblox/luau/pull/724
* Fixed https://github.com/Roblox/luau/issues/725

Co-authored-by: Arseny Kapoulkine <arseny.kapoulkine@gmail.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
2022-10-28 03:37:29 -07:00
JohnnyMorganz
ad7d006fb2
Use overloaded documentation symbol for class methods/table props (#724)
Right now `getDocumentationSymbol` on an overloaded class method or
table prop just gives the prop name, not prepended with `/overload/ +
toString(overload)`.

This causes the incorrect symbol to be used for `table.insert`,
`BrickColor.new`, `Color3.new` etc. (present in
https://raw.githubusercontent.com/MaximumADHD/Roblox-Client-Tracker/roblox/api-docs/en-us.json)

Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com>
2022-10-25 16:22:19 -07:00
Arseny Kapoulkine
e64d489e57
Update benchmark.yml
Add codegennull runs to compile time benchmarks
2022-10-24 17:24:19 -07:00
Andy Friesen
54324867df
Sync to upstream/release/550 (#723)
* Support `["prop"]` syntax on class definitions in definition files.
(#704)
* Improve type checking performance for complex overloaded functions
* Fix rare cases of incorrect stack traces for out of memory errors at
runtime
2022-10-21 10:54:01 -07:00
JohnnyMorganz
12ee1407a1
Add named parameters to math lib (#722)
Name the parameters used in `math` lib

This is mainly done to highlight the particular confusion for
`math.atan2`, where `y` comes before `x`, but this might not be
immediately obvious.

And then I added the rest of the names for consistency.

Note: I didn't add names to `math.random` as it's currently typed as
`(number?, number?) -> number`. Naming it `min` and `max` is technically
incorrect for the 1 argument version.
Maybe it should be typed as an intersection instead?
2022-10-21 08:05:56 -07:00
JohnnyMorganz
c4c120513f
Fix reverse deps > 1 node away not correctly marked as dirty (#719)
`Frontend.markDirty()` does not correctly mark reverse dependencies that
are not immediate as dirty.
This is because it would keep adding the reverse deps of `name` into the
queue to mark as dirty, instead of the reverse deps of `next`
2022-10-20 09:07:00 -07:00
Alan Jeffrey
662a5532ec
Minor fixes to the semantic subtyping blog post (#720) 2022-10-19 20:32:29 -05:00
Alan Jeffrey
ae5a011465
Add a blog post about semantic subtyping (#700)
Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com>
2022-10-18 16:56:15 -05:00
JohnnyMorganz
edb4453924
Support ["prop"] syntax on class definitions (#704)
Some classes have properties which are not valid identifiers (such as
https://create.roblox.com/docs/reference/engine/classes/Studio)

This adds support for the following syntax in definition files:
```lua
declare class Foo
    ["a property"]: string
end
```

Closes #702
2022-10-18 10:15:26 -07:00
Arseny Kapoulkine
49d6bc30ad
Add bench-codegen benchmark (#713)
We will now run luau with --codegen during benchmark runs and collect
the data into separate JSON. Note that we don't yet have the historical
data for these, which will be backfilled later.
2022-10-17 12:18:34 -07:00
Arseny Kapoulkine
c6a2d75193
Improve testing coverage with codegen/O2 in mind (#709)
This change adds codegen runs to coverage config and adds O2/codegen
testing to CI.

Note that we don't run O2 combinations in coverage - it's better that we
see gaps in O2 coverage in compiler tests, as these are valuable for
validating codegen intricacies that are difficult to see from
conformance tests passing/failing.
2022-10-17 10:02:21 -07:00
vegorov-rbx
76070f8da2
Sync to upstream/release/549 (#707)
* Reoptimized math.min/max/bit32 builtins assuming at least 2 arguments are used (1-2% lift on some benchmarks)
* Type errors that mention function types no longer have redundant parenthesis around return type
* Luau REPL now supports --compile=remarks which displays the source code with optimization remarks embedded as comments
* Builtin calls are slightly faster when called with 1-2 arguments (~1% improvement in some benchmarks)
2022-10-14 12:48:41 -07:00
JohnnyMorganz
ff736fd3e4
Fix segfault in loadDefinition for unit tests (#705)
`module` can be empty if the definition file has syntax errors
2022-10-14 07:28:54 -07:00